Logstash Pipeline Sequence

Hello Loggers,

I am wondering what happens when a Logstash thread get stuck at the input/filter-output level. Does Logstash create new threads to continue, or does it stay stuck indefinitely. I am asking this because I fed a 3.6 GB doc to Logstash, and I saw funny things happening. It took a while for around 700 message to go trough, then for about 10 minutes nothing else would get through Logstash. I decided to restart it, and about 1000 new messages went though (I believe these came from the SIGTERM flush). The rest of the messages never came through, but I think the rest of the messages did not come through because the when I restarted Logstash, the input threads were cut. Any thoughts?

https://www.elastic.co/guide/en/logstash/current/pipeline.html should help.

But if an input blocks then everything blocks as LS doesn't have any state machine internals.

1 Like