Batch size in logstash

I have 4 events having unique Id and I am processing these event in aggregate filter but with batch size of 250. Logstash is processing these events out of order I am not getting exact result, with batch size of 2 the events are processed in order but takes too much time

1 Like

The aggregate filter does impose some limitations and is not always appropriate. What is the high level business problem you are trying to solve?

I am having the same problem . Does logstash processes the events out of order within a particular batch of events?

I am trying to calculate time difference between two events from logs with aggregate filter, but within the same batch of events logstash processes events out of order and I am not getting the exact results always .

If pipeline.java_execution is enabled (which became the default in v7.0) then logstash will re-order events even with pipeline.workers set to 1.

2 Likes

Thank you for the reply . So Setting pipeline.java_execution :false will solve the problem of reordering right??

That is correct.

1 Like

Thank you that worked as expected .