Start warinnig

Elastic Stack Logstash
1

[2018-03-20T14:25:23,077][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>16, "pipeline.batch.size"=>12500, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>200000}
[2018-03-20T14:25:23,078][WARN ][logstash.pipeline ] CAUTION: Recommended inflight events max exceeded! Logstash will run with up to 200000 events in memory in your current configuration. If your message sizes are large this may cause instability with the default heap size. Please consider setting a non-standard heap size, changing the batch size (currently 12500), or changing the number of pipeline workers (currently 16)

how to change the pipeline.max_inflight"=>200000 it big than "pipeline.batch.size"=>12500,
how i can do it ?
thanks.

2

I don't understand what you're asking.

Logstash warns about pipeline.max_inflight for a reason. Why are you setting it so high? It's most likely a bad idea.

3

pipeline.max_inflight this i don't setting .
it auto output . i wan't to know how to change it pipeline.max_inflight to small.
thanks.
logstash is 5.6.8

4

in logstash.yml
i only setting this:

pipeline.batch.size: 12500
pipeline.batch.delay: 5
queue.max_events: 12500

but when start logstash the pipeline.max_inflight is output a warning.

5

Okay. I think pipeline.max_inflight is computed based on the batch size and the number of pipeline workers. You have 16 CPU cores, right? Or you've otherwise configured Logstash to have 16 pipeline
workers. 200,000 / 12,500 = 16.

6

ok i change the pipe work and try
thanks.

7

i add the workers in logstash
the warning is also.

[2018-03-21T08:44:35,443][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>16, "pipeline.batch.size"=>12500, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>200000}
[2018-03-21T08:44:35,443][WARN ][logstash.pipeline        ] CAUTION: Recommended inflight events max exceeded! Logstash will run with up to 200000 events in memory in your current configuration. If your message sizes are large this may cause instability with the default heap size. Please consider setting a non-standard heap size, changing the batch size (currently 12500), or changing the number of pipeline workers (currently 16)
8

???

You still have 16 pipeline workers so with a 12500 batch size the inflight event max is still 200k.

1 Like
9

how can do it ?
can't setting pipeline.batch.size: 12500 in logstash?

10

Reduce the batch size and/or the number of pipeline workers: https://www.elastic.co/guide/en/logstash/current/logstash-settings-file.html

Or, live with the warning and its consequences.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.