I have 3 logstash servers with 16GB memory and 4vcpu enabled in a cluster environment.
so my question is that what kind of size is better to set logstash pipline ?
My configure file like this as below:
Java heap size:
# This defaults to the number of the host's CPU cores. pipeline.workers: 4 # How many workers should be used per output plugin instance pipeline.output.workers: 4 # How many events to retrieve from inputs before sending to filters+workers pipeline.batch.size: 5000
[2017-07-31T11:59:45,748][WARN ][logstash.pipeline ] CAUTION: Recommended inflight events max exceeded! Logstash will run with up to 20000 events in memory in your current configuration. If your message sizes are large this may cause instability with the default heap size. Please consider setting a non-standard heap size, changing the batch size (currently 5000), or changing the number of pipeline workers (currently 4)
Ａny ideas of what kind of size is better to set logstash pipline ?
by the way, what's max pipline size of single instance logstash even if with a big memory or multiple core cpus?
Thanks in adavance