Logstash Windows Service - error java.lang.IllegalStateException

[Scott_Newby]

Logstash continues to restart repeating -

[ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

I've tried: removing the config.reload.automatic = true setting in logstash.yml, setting the service account to localsystem and my personal account. Ensured that I have permissions to the config file in logstash...

Any help would be appreciated.

[Scott_Newby]

Along with this behavior, when the service account is set to my account I get AppData\Local\Temp continually creating jruby-NNNNN directories and jffiNNNNNN.dll files being created.

[magnusbaeck (Magnus Bäck)]

Is the quoted error message the only message in the log? It's usually just the final message before Logstash quits, with the interesting messages further up.

[Scott_Newby]

Magnus - thanks for the quick response...

[2018-08-03T10:32:56,001][WARN ][logstash.runner ] SIGINT received. Shutting down. [2018-08-03T10:32:56,206][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_0.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_0.json>) [2018-08-03T10:32:56,232][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_1.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_1.json>) [2018-08-03T10:32:56,233][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_0.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_0.json>) [2018-08-03T10:32:56,233][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_1.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_1.json>) [2018-08-03T10:32:56,795][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>80, "name"=>"[CucumberAggregation]<file", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}], ["LogStash::Filters::Mutate", {"remove_field"=>"message", "id"=>"288ba2c0b613390e0984968d7be0798d7a7a12ad9dec1386740a79404e6befe1"}]=>[{"thread_id"=>76, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:in read_batch'"}, {"thread_id"=>77, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:inread_batch'"}, {"thread_id"=>78, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:in read_batch'"}, {"thread_id"=>79, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/pipeline.rb:418:inread_batch'"}]}}
[2018-08-03T10:32:57,357][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_0.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_0.json>)
[2018-08-03T10:32:57,358][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_1.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastStableBuild/cucumber-html-reports/.cache/report_1.json>)
[2018-08-03T10:32:57,358][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_0.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_0.json>)
[2018-08-03T10:32:57,359][ERROR][logstash.inputs.file ] each: active?: /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_1.json: (#<Errno::ESRCH: No such process - /Jenkins/jobs/Testing Pipeline/jobs/Pipeline/builds/lastSuccessfulBuild/cucumber-html-reports/.cache/report_1.json>)
[2018-08-03T10:35:38,405][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
[2018-08-03T10:36:19,968][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
[2018-08-03T10:36:52,951][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
[2018-08-03T10:37:24,464][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
[2018-08-03T10:37:56,028][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit`

The last doc loaded on time stamp was from 8-3-2018 14:12Z time and the prior to the j.l.illegalStateException generally repeats similarly throughout the prior logs.

[magnusbaeck (Magnus Bäck)]

The first log message ("SIGINT received") indicates that Logstash somehow has received a signal to stop. Since Windows doesn't have signals I don't know what would result in a SIGINT being received. The error messages that follow are probably related to the SIGINT signal and might be normal on a platform that doesn't have signals. This is a bit too Windows-specific for me.