Some problem of logstash elapsed filter


I use elapsed filter to calculate events duration, but it has something wrong. The elapsed_time got negative number in my case.

config below:

Can someone help me?

(Lewis Barclay) #2

Your start and end events coming in very quickly, 100ms from each other is that correct?


Thanks for your reply first. I got a search result for elpased_time between 0 to 1, like below pic.

Sometime elapsed get work when events between 100ms. But it maybe duration of events too fast to got this problem. And, I want to let it become normally to work correct. Hope for more discuss. Thanks.

(Lewis Barclay) #4

Correct I think possibly the events are too fast


Have other solution?I want to use elapsed to do performance report.

(Lewis Barclay) #6

I am unsure, perhaps raise a bug on github with the logstash team?

(Christian Dahlqvist) #7

How many worker threads do you have configured for Logstash?


It has default setting, but I think it have 4 threads.

(Christian Dahlqvist) #9

If I recall correctly, one of the major drawbacks with the elapsed and aggregate filters is that all related events have to pass through the same processing thread, thereby limiting Logstash to a single worker thread and instance. This naturally severely limits throughput and scales very badly. Try setting the number of worker threads to 1 to see if it resolves the accuracy issue.

A more scalable approach may be to have a periodic batch job that post-process events once they are in Elasticsearch, but although this is likely to scale better it does introduce a delay.


There are some wrong elapsed time after change the thread to 1 worker. Changing thread maybe let it work, but still have some wrong elapsed.

Ooes pipeline.batch.size and pipeline.batch.delay have to change?

(Christian Dahlqvist) #11

What does the underlying events that contributed to an incorrect elapsed time look like if you view them in Kibana?


I take a few events picture. Its tag seem like missing start_tag.

Do my elapsed code wrong in logstash.conf? Doesn't have to use else if?


Hello, who can tell me where is going wrong?

(system) closed #14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.