Logstash - Unknown setting 'ssl_certificate' & 'ssl_key' for elasticsearch

Elastic Stack Logstash
1

Hi guys,
I am trying to connect logstash with elasticsearch that has security enabled. While elasticsearch is running well with the SSL Certificates/keys, the same certificates/keys are not working for the Logstash. Following is my configuration and error.

  1. My output.conf file.
output {
        file {
                create_if_deleted => true
                path => "/var/log/logstash/logstash_log"
                codec => "rubydebug"
        }
        elasticsearch {
                hosts => ["https://localhost:9200"]
                user => "admin"
                password => "admin"
                manage_template => false
                ssl => true
                cacert => "/etc/elasticseach/root-ca.pem"
                ssl_certificate => "/etc/elasticsearch/esnode.pem"
                ssl_key => "/etc/elasticsearch/esnode-key.pem"
                index => "%{[index_name]}-%{+YYYY.MM.dd}"
                document_id => "%{[@metadata][fingerprint]}"
                http_compression => true
        }
}
  1. Error in logs
Nov 27 19:29:31 [localhost] logstash: [2020-11-27T19:29:31,573][ERROR][logstash.outputs.elasticsearch] Unknown setting 'ssl_certificate' for elasticsearch
Nov 27 19:29:31 [localhost] logstash: [2020-11-27T19:29:31,578][ERROR][logstash.outputs.elasticsearch] Unknown setting 'ssl_key' for elasticsearch
  1. My Elasticsearch version is 7.9.1 and Logstash version is 7.9.1

Any help is appreciated. Thank you.

2

Delete these lines, the elasticsearch output does not support them.

3

Oh! Okay.
But how should I add SSL certificates to the output conf?
Because I have already added this, cacert is working but not rest...

               cacert => "/etc/elasticseach/root-ca.pem"
               ssl_certificate => "/etc/elasticsearch/kirk.pem"
               ssl_key => "/etc/elasticsearch/kirk-key.pem"

Because without SSL ssl => true, following error is showing.

Nov 27 19:56:25 [localhost] logstash: [2020-11-27T19:56:25,356][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://admin:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://admin:xxxxxx@localhost:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
4

That would suggest the cacert is not correct.

5

After removing the two lines mentioned, now it is taking cacert in account but Logstash service failing due to following reasons.

Nov 27 20:26:10 [localhost] logstash: LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: Got response code '500' contacting Elasticsearch at URL 'https://localhost:9200/_xpack'

Nov 27 20:26:11 [localhost] logstash: [2020-11-27T20:26:11,009][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError: LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError>,...
6

Do the elasticsearch logs contain anything that explains the 500 response?

7

Yes. It was another issue. Actually I am using Open distro for Elasticsearch, so it does not x-pack in it and Logstash was attempting to connect to its /_xpack endpoint. That is why it was failing with 500 error code. So I resolved that issue using this method.
Thank you for your assistance @Badger.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.