Logstash version: 5.6.7,
Java version: 1.8.0_161
After no notable changes, Logstash was missing logs a couple mornings ago. In making some changes to the pipeline config, I restarted the service and it did not come back up. After substantial debugging, I limited my pipeline to the following config, and Logstash still would not start:
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["my_es_node:9200"]
sniffing => true
manage_template => false
index => "filebeat-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
However I consistently get the following error:
[2018-02-01T10:49:03,069][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>6, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>750}
[2018-02-01T10:49:03,224][DEBUG][io.netty.util.internal.logging.InternalLoggerFactory] Using SLF4J as the default logging framework
[2018-02-01T10:49:03,418][ERROR][logstash.pipeline ] Error registering plugin {:plugin=>"<LogStash::Inputs::Beats port=>5044, id=>\"2aa5ab800c31d0d28119167a679a06e8cec77176-1\", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>\"plain_227478cc-9477-4220-a478-c6f2c9a30bb6\", enable_metric=>true, charset=>\"UTF-8\">, host=>\"0.0.0.0\", ssl=>false, ssl_verify_mode=>\"none\", include_codec_tag=>true, ssl_handshake_timeout=>10000, congestion_threshold=>5, target_field_for_codec=>\"message\", tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>[\"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256\"], client_inactivity_timeout=>60, executor_threads=>24>", :error=>"cannot link Java class io.netty.handler.ssl.OpenSsl, probable missing dependency: io/netty/handler/ssl/OpenSslEngine"}
[2018-02-01T10:49:03,428][DEBUG][logstash.pipeline ] Pushing shutdown {:thread=>"#<Thread:0x61410100 sleep>"}
[2018-02-01T10:49:03,428][DEBUG][logstash.pipeline ] Pushing shutdown {:thread=>"#<Thread:0x19162d1b sleep>"}
[2018-02-01T10:49:03,428][DEBUG][logstash.pipeline ] Pushing shutdown {:thread=>"#<Thread:0x16c81542 sleep>"}
[2018-02-01T10:49:03,428][DEBUG][logstash.pipeline ] Pushing shutdown {:thread=>"#<Thread:0x61cdc70 run>"}
[2018-02-01T10:49:03,429][DEBUG][logstash.pipeline ] Pushing shutdown {:thread=>"#<Thread:0x6d686303 sleep>"}
[2018-02-01T10:49:03,429][DEBUG][logstash.pipeline ] Pushing shutdown {:thread=>"#<Thread:0x71d3fd06 sleep>"}
[2018-02-01T10:49:03,429][DEBUG][logstash.pipeline ] Shutdown waiting for worker thread #<Thread:0x61410100>
[2018-02-01T10:49:03,472][DEBUG][logstash.pipeline ] Shutdown waiting for worker thread #<Thread:0x19162d1b>
[2018-02-01T10:49:03,473][DEBUG][logstash.pipeline ] Shutdown waiting for worker thread #<Thread:0x16c81542>
[2018-02-01T10:49:03,478][DEBUG][logstash.pipeline ] Shutdown waiting for worker thread #<Thread:0x61cdc70>
[2018-02-01T10:49:03,478][DEBUG][logstash.pipeline ] Shutdown waiting for worker thread #<Thread:0x6d686303>
[2018-02-01T10:49:03,478][DEBUG][logstash.pipeline ] Shutdown waiting for worker thread #<Thread:0x71d3fd06>
[2018-02-01T10:49:03,479][DEBUG][logstash.outputs.elasticsearch] closing {:plugin=>"LogStash::Outputs::ElasticSearch"}
[2018-02-01T10:49:03,479][DEBUG][logstash.outputs.elasticsearch] Stopping sniffer
[2018-02-01T10:49:04,065][DEBUG][logstash.outputs.elasticsearch] Stopping resurrectionist
[2018-02-01T10:49:04,066][DEBUG][logstash.outputs.elasticsearch] Waiting for in use manticore connections
[2018-02-01T10:49:04,066][DEBUG][logstash.outputs.elasticsearch] Closing adapter #<LogStash::Outputs::ElasticSearch::HttpClient::ManticoreAdapter:0x774cab9c>
[2018-02-01T10:49:04,081][ERROR][logstash.agent ] Pipeline aborted due to error {:exception=>#<NameError: cannot link Java class io.netty.handler.ssl.OpenSsl, probable missing dependency: io/netty/handler/ssl/OpenSslEngine>, :backtrace=>["org/jruby/javasupport/JavaClass.java:204:in `for_name'", "org/jruby/javasupport/JavaUtilities.java:34:in `get_proxy_class'", "file:/usr/share/logstash/vendor/jruby/lib/jruby.jar!/jruby/java/core_ext/object.rb:27:in `java_import'", "org/jruby/RubyArray.java:2414:in `map'", "file:/usr/share/logstash/vendor/jruby/lib/jruby.jar!/jruby/java/core_ext/object.rb:22:in `java_import'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-3.1.26-java/lib/logstash/inputs/beats.rb:140:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:290:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:301:in `register_plugins'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:301:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:456:in `start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:348:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:235:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:408:in `start_pipeline'"]}
Specifically this error:
:error=>"cannot link Java class io.netty.handler.ssl.OpenSsl, probable missing dependency: io/netty/handler/ssl/OpenSslEngine"