Sequence number for ECS Events received by TCP?

Elastic Stack Beats
,
1

Hi,

I have configured my application server (JBoss) to format log messages using the ECS formatter and directly send them to Filebeat via TCP.

This works mostly fine - however, there is one issue: Log entries having the same timestamp might appear in random order because apparently no sequence number is being written.

Any idea how to fix that?

Thank you!

2

This may impact performance of filebeat, but you can try setting the number of processors with the max_procs setting to 1. This may force entries to be process in the order that they are received without any sort of multithreading, which is what I suspect is happening.

1 Like
3

Thanks for the reply!

The order of the log entries in the log view is only part of the issue, though. At some points, I also want to export log streams back to file based logs (e.g. if some third party requests a log to diagnose an issue). For this, I am using Elasticsearch’s search API which requires sorting the events before export in order to export more than 10,000 events.

Thus, I really need a sequence number as a second sort criteria to uniquely identify events.

If I force FileBeat to process events single-threaded: Is there any processor available which I can use to add a sequence number as a field?
Or can I rely on the event IDs (or any other field) generated by FileBeat to be strictly monotonically increasing (at least for most cases in which timestamps are the same)?

If not: Should the ECS formatter component maybe be responsible for generating the sequence number instead? Not sure where the responsibility should lie for this.

4

Just to double down on that one:
I just realized that the LogRecord class already provides a method getSequenceNumber. However, the EcsFormatter ignores this field.

I have created an issue in GitHub to request this feature: Write SequenceNumber in jboss-logmanager-ecs-formatter · Issue #154 · elastic/ecs-logging-java · GitHub

5

Good catch!
I thought that maybe using the UUID filter from logstash would be a good solution as that generates a unique ID for each event (no matter the content of the event because it uses time as part of the generation) and you may be able to sort by that field on output. However, it's not something I've tried before.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.