Centralised Pipeline Management reload error


(Jiří Bednář) #1

Greetings

I just upgraded my ELK stack to 6.5.0, and 4 out of my 6 logstash nodes fail to load 1 out of 5 pipelines. The config is the same, and the pipeline is centrally managed, so I don't see why only some nodes are affected.
Here are the relevant log entries produced when I try to manually reload the pipeline:

[2018-11-15T12:28:43,053][INFO ][logstash.pipelineaction.reload] Reloading pipeline {"pipeline.id"=>:clog_dovecot}
[2018-11-15T12:28:43,061][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Reload/pipeline_id:clog_dovecot, :exception=>"NoMethodError", :message=>"undefined method `call' for nil:NilClass", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-redis-3.4.0/lib/logstash/inputs/redis.rb:110:in `stop'", "/usr/share/logstash/logstash-core/lib/logstash/inputs/base.rb:87:in `do_stop'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:464:in `stop_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:447:in `shutdown'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:14:in `block in execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:13:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:53:in `block in execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:317:in `block in converge_state'"]}
[2018-11-15T12:28:48,205][INFO ][logstash.pipelineaction.reload] Reloading pipeline {"pipeline.id"=>:clog_dovecot}
[2018-11-15T12:28:48,211][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Reload/pipeline_id:clog_dovecot, :exception=>"NoMethodError", :message=>"undefined method `call' for nil:NilClass", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-redis-3.4.0/lib/logstash/inputs/redis.rb:110:in `stop'", "/usr/share/logstash/logstash-core/lib/logstash/inputs/base.rb:87:in `do_stop'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:464:in `stop_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:447:in `shutdown'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:14:in `block in execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/stop.rb:13:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:53:in `block in execute'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:317:in `block in converge_state'"]}

Restarting the affected nodes doesn't help either. How can I fix this issue?


(Jiří Bednář) #2

My apologies, there was an inconsistency between files in the /elc/logstash/patterns.d/ directory.
Is there a way to use the centralised pipeline management to synchronise the grok patterns also?


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.