Monitoring for logstash 6.3 cannot be used


(BallGhost) #1

Recently I upgraded to ELK6.3 from 6.2.2,but the monitoring for logstash could not be displayed on kibana.

The monitoring for ES and kibana is normal.

Here is the logstash.yml
"""
...
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: ***
xpack.monitoring.elasticsearch.password: ***
xpack.monitoring.elasticsearch.url: ***
...
"""

Below is the logstash's error level log

[ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>".monitoring-logstash", :plugin=>"#LogStash::OutputDelegator:0x7592752c", :error=>"Unrecognized SSL message , plaintext connection?", :thread=>"#<Thread:0x321399ed run>"}
[ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>".monitoring-logstash", :exception=>#<Manticore::UnknownException: Unrecognized SSL message, plaintext connection?>, :backtrace=>["/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/response.rb:37:in block in initialize'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/response.rb:79:incall'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/manticore_adapter.rb:74:in perform_request'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:291:inperform_request_to_url'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:245:in block in healthcheck!'", "org/jruby/RubyHash.java:1343:ineach'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:241:in healthcheck!'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:341:inupdate_urls'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:71:in start'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:299:inbuild_pool'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client.rb:63:in initialize'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:increate_http_client'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:98:in build'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch.rb:231:inbuild_client'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.1.1-java/lib/logstash/outputs/elasticsearch/common.rb:24:in register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:97:inregister'", "org/logstash/config/ir/compiler/OutputDelegatorExt.java:93:in register'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:340:inregister_plugin'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:351:in block in register_plugins'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:351:in register_plugins'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:728:inmaybe_setup_out_plugins'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:361:in start_workers'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:288:inrun'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:248:in `block in start'"], :thread=>"#<Thread:0x65341c run>"}
[2018-06-27T14:56:28,899][ERROR][logstash.agent ] Failed to execute action {:id=>:".monitoring-logstash", :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<.monitoring-logstash>, action_result: false", :backtrace=>nil}

In addition, the index like .monitoring-logstash* does not exist, and I am using the base license

Please help me, Thanks


(Ioannis Kakavas) #2

Looks like TLS is enabled for the HTTP layer in Elasticsearch but Logstash is attempting to connect over plain http.

  • Is that an http URL ?

  • Is TLS enabled in Elasticsearch for the http layer ? ( you can share the relevant part of your elasticsearch.yml )


(BallGhost) #3

As you said, “xpack.monitoring.elasticsearch.url” is https://***
I just changed it to http, now the monitoring for logstash works properly ,and the error log is gone.
Thank you !!


(Ioannis Kakavas) #4

Glad to hear. I actually misread the error message so :

should have been

Looks like TLS is not enabled for the HTTP layer in Elasticsearch but Logstash is attempting to connect over https.


(BallGhost) #5

Haha, this is a small qustion. Thanks again


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.