Specifying Logstash filter workers?

Elastic Stack Logstash
1

I am currently trialling the use of the Aggregated Filter plugin within Logstash v7.7 for a specific use case. I noticed that the description the specific Plugin (here) says:

You should be very careful to set Logstash filter workers
to 1 ( -w 1 flag) for this filter to work correctly otherwise
events may be processed out of sequence and unexpected
results will occur.

I have already looked through everything under /etc/logstash and could not find anywhere where this could be specified? The closest I got was pipeline.workers in /etc/logstash/logstash.yml which is accompanied by the comment

Set the number of workers that will, in parallel, execute the filters+outputs stage of pipelines.

This defaults to the number of hosts's CPU cores.

Don't suppose this is it?

PS: I have already looked through this fairly dated thread.

2

You can set it in logstash.yml. You can set it per-pipeline in pipelines.yml, or you can set it on the command line using -w 1

In 7.7 you will also need to disable java execution to preserve the order of events.

3

Thanks! Could you please point me to the docs/ link that specifies which config to set?

Also, TA for the disable java execution pointer :vulcan_salute:

4

Docs for the command line, logstash.yml, and pipelines.yml.

5

Ah, thanks! :slight_smile:

The command line docs confirmed that -w is the same as pipeline.workers. And it can be set in both, logstash.yml and pipelines.yml (per-pipeline).

And also, that disabling java execution can be done by setting:

pipeline.java_execution: false # default true
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.