Logstash pipeline.workers

Elastic Stack Logstash
1

Hello!

Currently we have our pipeline.yml without defined workers. We understand that we take by default the number of cpu cores of the machine.

Should we define all in pipeline.worker: 1 ?
How do we decide which workers have to have each pipeline ?

Thanks in advance !

2

Generally you want the total number of active threads to be equal to the number of CPUs. This avoids context switching, which is expensive.

If you have a single pipeline on a server dedicated to logstash then the default of having the number of worker threads equal to the number of CPUs makes perfect sense.

If you have a server that runs multiple apps then decide how many of the CPUs you want logstash to use, and then allocate them among the pipelines that you have. It is OK to have slightly more worker threads than CPUs, but not significantly more.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.