Hello,
I have been using logstash for some time now and I recently upgraded from an old version (5.3.2) to a newer one (6.2.4). Using basically the same parameters I noticed a dramatic decrease in the rate of processed records. I decided to make some tests on a machine using as input a file and writing also to another file on disk. Changing parameters and memory only meant variations of few records/s but the logstash version really made a difference.
logstash 5.3.2: ~85/s
logstash 6.2.4: ~30/s
logstash 7.7.0: ~15/s
My application involves the recollection of pieces of information scattered among various log lines and thus I'm using the logstash-filter-aggregate plugin with a single output pipeline. And, just in case that it also matters, I'm also accessing an sqlite3 DB to append some information obtained from other sources (ruby code within the filter).
Please, could you comment on this and tell me whether this might be expected ? Thanks very much for any helpful advice.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.