Why is Logstash stalling for so long?

Elastic Stack Logstash
1

I have a pipeline set up to an S3 bucket that has ~700k files in it. When I restart a pipeline (I have auto-restart enabled) it takes hours to restart. Looking at Logstash's logs I see huge blocks of

[2018-04-05T14:30:20,701][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:30:25,702][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:30:30,702][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:30:35,701][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:30:40,701][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:30:45,702][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:30:50,702][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:30:55,701][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:31:00,701][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:31:05,701][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}
[2018-04-05T14:31:10,702][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>5, "stalling_thread_info"=>{}}

It started out with inflight_count in the hundreds, but even when it gets down to 5, I don't know why it is taking so long to just stop these threads.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.