How to define metric's @timestamp from Data collection side(which is beat side)?

Elastic Stack Logstash
1

Dear folks,

Recently I've been working on customizing beat with bash to generate the json data that logstash needed instead of to write own beat with go.

Cuz I'm not the go expert. I'm good to implement customizing beat with script.
I didn't add field @timestamp for original json raw data gernated from script,after the json raw data sent to logstash, logstash addes some metadata to this raw json data (like @version ,@timestamp).

The problem is that, I want to add the field @timestamp to original json raw data ,not want to let logstash or elasticsearch to add or modify @timestamp.
otherwise the metrics we collected that script triggered is not precise at that time.

By the way, how linux "date" command generate the correct the unix timestamp based iso8601?

example:

"@timestamp": "2017-09-15T03:19:11.610Z"

(I think this timestamp is Accurate to milliseconds and end up with Three decimal places)

Any suggestions or comments are grateful?

2

By the way, how linux "date" command generate the correct the unix timestamp based iso8601?

date -u +%Y-%m-%dT%H:%M:%SZ gives you the right format but without the milliseconds. I don't think date(1) supports millisecond resolution.

Generating JSON files using bash doesn't strike me as a great idea. I'd use a real language that gives correct JSON serialization and supports millisecond resolution for timestamps without jumping through hoops.

3

@magnusbaeck

Ｔhanks for your prompt reply, you're right.
but what language would you like to recommend for customizing the metrics with Jason serialization.

4

but what language would you like to recommend for customizing the metrics with Jason serialization.

Any language that you're comfortable with that can serialize JSON.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.