Hello.
I have a setup with a log source that is sending about 4k EPS and i have an HAProxy that is receiving those messages and sending them to currently only one Logstash machine.
That machine has 8 cores and 8GB of RAM. Logstash in that machine is setup to parse that log source with a bunch of operations that amount to if's and mutates. No plugins out of standard logstash distribution.
The problem is no matter the settings I put in pipeline.workers, i can not get the single logstash instance to handle the 4k EPS. The most I can get is about 2.2k EPS.
But when i start another instance of Logstash in the same machine with the same settings, the aggregate EPS handled by them both then sums to 4k EPS.
Both logstash instances running on that machine share the same pipeline configuration with the exception of the input port.
The machine is a VM running Ubuntu 18.04 LTS with kernel 4.15.0-101-generic. Logstash is 7.9.
How can this be ? Am I hitting some limitation of the JVM ?
Regards,
Ricardo Ferreira