[Filebeat] Timestamp with nanosecond precision

Elastic Stack Beats
1

My configuration:

  • filebeat + logstash + elasticsearch + kibana (with logtrail) - all in version 7.6.1

I have a mule.log file which contains information without the timestamp field that I could extract to enforce proper order. Snippet from mule.log:

 --> Wrapper Started as Console
Launching a JVM...
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using the ParNew young collector with the Serial old collector is deprecated and will likely be removed in a future release
Starting the Mule Container...

The problem I have is that current precision in @Timestamp field causes that f.e. lines 1-3 have 2020-06-14T10:04:39.249Z and lines 4-6 have 2020-06-14T10:04:39.250Z. So both group lines (1-3 and 4-6) are out of order in logtrail. For exmaple lines 1-3 appear (f.e.):

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Launching a JVM...
--> Wrapper Started as Console

instead of:

--> Wrapper Started as Console
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Launching a JVM...

The same goes for lines 4-6 (and others with the same @Timestamp value).

  1. I see that it supposed to be already handled, but either I'm doing somethin wrong (hence the question) or it's only for docker files.
  2. I could base order of each "event"/"line"/"message" on the offset field but that is not a clean solution.

How can I mark each filebeat event, in orde to be able to keep the proper order of events ? (in my case in logtrail)

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.