We use central pipeline management for logstash. After pushing incorrect config (should be string, but was long),
Logstash enters a loop where it tries to restart the plugin
- Logstash does not download the new config and restart the pipeline
- Logstash does not respond to systemctl stop command
Only kill -9 works
Expected behavoiur is that logstash would detect a new config and restart the pipeline.
We use version 6.6.1.
Awesome for any input on this!
[2019-03-08T11:16:58.961+0100][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:in.metrics.netx.conf
Plugin: <LogStash::Inputs::Kafka enable_auto_commit=>"true",
-- Removed config values
Error: Invalid value 100 for configuration max.poll.records: Expected value to be a 32-bit integer, but it was a java.lang.Long
Exception: Java::OrgApacheKafkaCommonConfig::ConfigException
Stack: org.apache.kafka.common.config.ConfigDef.parseType(org/apache/kafka/common/config/ConfigDef.java:669)
org.apache.kafka.common.config.ConfigDef.parseValue(org/apache/kafka/common/config/ConfigDef.java:471)
org.apache.kafka.clients.consumer.KafkaConsumer.(org/apache/kafka/clients/consumer/KafkaConsumer.java:632)
java.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:423)
org.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:278)
org.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:1001)
org.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1.lib.logstash.inputs.kafka.invokeOther135:new(usr/share/logstash/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1/lib/logstash/inp
uts//usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.3.1/lib/logstash/inputs/kafka.rb:328)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1.lib.logstash.inputs.kafka.create_consumer(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.3.1/lib/logstash/inputs/kafka.rb:328)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1.lib.logstash.inputs.kafka.invokeOther3:create_consumer(usr/share/logstash/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1/lib/logstash/inputs//usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.3.1/lib/logstash/inputs/kafka.rb:226)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1.lib.logstash.inputs.kafka.block in run(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.3.1/lib/logstash/inputs/kafka.rb:226)
org.jruby.RubyEnumerable$23.call(org/jruby/RubyEnumerable.java:846)
org.jruby.RubyFixnum.times(org/jruby/RubyFixnum.java:305)
org.jruby.RubyFixnum$INVOKER$i$0$0$times.call(org/jruby/RubyFixnum$INVOKER$i$0$0$times.gen)
org.jruby.RubyEnumerable$INVOKER$s$0$0$map.call(org/jruby/RubyEnumerable$INVOKER$s$0$0$map.gen)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1.lib.logstash.inputs.kafka.invokeOther9:map(usr/share/logstash/vendor/bundle/jruby/$2_dot_3_dot_0/gems/logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1/lib/logstash/inputs//usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.3.1/lib/logstash/inputs/kafka.rb:226)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_kafka_minus_8_dot_3_dot_1.lib.logstash.inputs.kafka.run(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-kafka-8.3.1/lib/logstash/inputs/kafka.rb:226)
RUBY.inputworker(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:426)
RUBY.block in start_input(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:420)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:289)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:246)
java.lang.Thread.run(java/lang/Thread.java:748)
[2019-03-08T11:16:58.961+0100][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:in.logs.nlpp.conf
Plugin: <LogStash::Inputs::Kafka enable_auto_commit=>"true">
--- Removed config values
Error: Invalid value 100 for configuration max.poll.records: Expected value to be a 32-bit integer, but it was a java.lang.Long