Logstash with NSSM never ending when stop service

Elastic Stack Logstash
1

Hello,

I use NSSM for create a Windows Service to start/stop Logstash. I follow the instruction you can find here https://github.com/verbosemode/public-notes/blob/master/logstash-windows.md

The Windows Service is create correctly and i haven't any trooble to start Logstash.

When I stop the service, Logstash is steel parsing information from jdbc input. I can find in logfile the SQL statement is still running and in task management I have a java.exe running with logstash parameters.

I try to start logstash with the option pipeline.unsafe_shutdown: true in logstash.yml but I have always this problem.

You can find my jdbc parameters I use in logstash-parser.conf :

    jdbc {
		jdbc_driver_library => "${ELK_JDBC_LIB}"
		jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
		jdbc_connection_string => "jdbc:${ELK_JDBC_CHAINE}"
		jdbc_user => "${ELK_JDBC_USER}"
		jdbc_password => "${ELK_JDBC_PWD}"

		statement_filepath => "${ELK_JDBC_QUERY}"
		schedule => "*/2 * * * *"
		type => esb_bdd

	}

Have you any idea to solve that, pls ?

2

To be complet, logs when I stop Logstash service

Terminer le programme de commandes (O/N) ? 17:51:23.963 [SIGINT handler] WARN logstash.runner - SIGINT received. Shutting down the agent.
17:51:24.010 [LogStash::Runner] WARN logstash.agent - stopping pipeline {:id=>"main"}
17:51:28.963 [Ruby-0-Thread-39: E:/ELK/logstash-5.4.0/logstash-core/lib/logstash/runner.rb:409] WARN logstash.runner - Received shutdown signal, but pipeline is still waiting for in-flight events
to be processed. Sending another ^C will force quit Logstash, but this may cause
data loss.
17:51:29.245 [Ruby-0-Thread-40: E:/ELK/logstash-5.4.0/logstash-core/lib/logstash/shutdown_watcher.rb:31] WARN logstash.shutdownwatcher - {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>36, "name"=>"[main]<stdin", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/logstash-input-stdin-3.2.2/lib/logstash/inputs/stdin.rb:77:in read'"}, {"thread_id"=>37, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>38, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>39, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>40, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watched_file.rb:142:in initialize'"}, {"thread_id"=>41, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>42, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>43, "name"=>"[main]<jdbc", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler.rb:170:injoin'"}], ["LogStash::Filters::Mutate", {"replace"=>{"type"=>"%{new_type}"}, "remove_field"=>"new_type", "id"=>"8e89a1c61f91ae7e37c255fcec3d6bc02d7f2c94-29"}]=>[{"thread_id"=>35, "name"=>"[main]>worker0", "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:136:in synchronize'"}]}} 17:51:29.245 [Ruby-0-Thread-40: E:/ELK/logstash-5.4.0/logstash-core/lib/logstash/shutdown_watcher.rb:31] ERROR logstash.shutdownwatcher - The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information. 17:51:34.057 [Ruby-0-Thread-40: E:/ELK/logstash-5.4.0/logstash-core/lib/logstash/shutdown_watcher.rb:31] WARN logstash.shutdownwatcher - {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>36, "name"=>"[main]<stdin", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/logstash-input-stdin-3.2.2/lib/logstash/inputs/stdin.rb:77:inread'"}, {"thread_id"=>37, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>38, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>39, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>40, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>41, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>42, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>43, "name"=>"[main]<jdbc", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler.rb:170:in join'"}], ["LogStash::Filters::Mutate", {"replace"=>{"type"=>"%{new_type}"}, "remove_field"=>"new_type", "id"=>"8e89a1c61f91ae7e37c255fcec3d6bc02d7f2c94-29"}]=>[{"thread_id"=>35, "name"=>"[main]>worker0", "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:136:insynchronize'"}]}}
17:51:39.213 [Ruby-0-Thread-40: E:/ELK/logstash-5.4.0/logstash-core/lib/logstash/shutdown_watcher.rb:31] WARN logstash.shutdownwatcher - {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>36, "name"=>"[main]<stdin", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/logstash-input-stdin-3.2.2/lib/logstash/inputs/stdin.rb:77:in read'"}, {"thread_id"=>37, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>38, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>39, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>40, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>41, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:insleep'"}, {"thread_id"=>42, "name"=>"[main]<file", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/filewatch-0.9.0/lib/filewatch/watch.rb:288:in sleep'"}, {"thread_id"=>43, "name"=>"[main]<jdbc", "current_call"=>"[...]/vendor/bundle/jruby/1.9/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler.rb:170:injoin'"}], ["LogStash::Filters::Mutate", {"replace"=>{"type"=>"%{new_type}"}, "remove_field"=>"new_type", "id"=>"8e89a1c61f91ae7e37c255fcec3d6bc02d7f2c94-29"}]=>[{"thread_id"=>35, "name"=>"[main]>worker0", "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:136:in `synchronize'"}]}}
17:51:39.213 [Ruby-0-Thread-40: E:/ELK/logstash-5.4.0/logstash-core/lib/logstash/shutdown_watcher.rb:31] FATAL logstash.shutdownwatcher - Forcefully quitting logstash..

After that, at 17:52 I have a new line whith my SQL Statement

17:52:00.104 [Ruby-0-Thread-38: E:/ELK/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:283] INFO logstash.inputs.jdbc - (0.000000s) select inst.id_instance_flux as idInstanceFlux, fc.id_projet_flux as idProjetFlux, f.code as codeFlux, c.code as codeClient, a.code as codeAppli, re.libelle as libelleErreur
, inst.timestamp_debut, inst.timestamp_fin, inst.statut, inst.declenchement_flux, convert(varchar,timestamp_debut,121) as timestamp_elk
from instance_flux inst....

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.