Logstash arbitrarily hangs with SIGINT. Happens roughly every few days

Elastic Stack Logstash
1

We have a Logstash server as part of our Elastic Stack on Windows. About once a week, logs stop ingesting and the culprit is always Logstash. Restarting the service on that VM fixes the issue, but I'd like to find the proper solution that prevents this outright.

The last section of the Logstash internal log is as follows:

[2021-12-08T11:32:58,409][WARN ][logstash.runner          ] SIGINT received. Shutting down.
[2021-12-08T11:33:01,487][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline terminated {"pipeline.id"=>".monitoring-logstash"}
[2021-12-08T11:33:03,496][WARN ][logstash.runner          ] Received shutdown signal, but pipeline is still waiting for in-flight events
to be processed. Sending another ^C will force quit Logstash, but this may cause
data loss.
[2021-12-08T11:33:06,889][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{}}
[2021-12-08T11:33:06,936][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2021-12-08T11:33:12,014][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{}}
[2021-12-08T11:33:17,098][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{}}
[2021-12-08T11:33:18,959][INFO ][logstash.javapipeline    ][log] Pipeline terminated {"pipeline.id"=>"log"}
[2021-12-08T11:33:19,830][INFO ][logstash.runner          ] Logstash shut down.

The VM Logstash runs on has no other users, shares no functionality. Uncertain what could be sending it a SIGINT causing this.

Any suggestions on some things to check?

2

In my experience this happens when one of your logstash pipelines is waiting on something. This is usually because of a plugin is waiting for some sort of a response from another system or a filter like aggregation or elapsed is waiting for a timeout.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.