TLS/SSL for logstash 5.6

All,

We have tried by adding TLS CA certificate which we already generated for ES cluster.

When we add the parameters in logstash configuration file as per below doc

https://www.elastic.co/guide/en/x-pack/5.6/logstash.html

We see errors in log files and logstash could not able to establish the pipeline. Below are the errors

[2019-02-17T05:19:41,166][ERROR][logstash.outputs.elasticsearch] Invalid setting for elasticsearch output plugin:

  output {
    elasticsearch {
      # This setting must be a path
      # File does not exist or cannot be opened /etc/logstash/certs/cert.pem
      cacert => "/etc/logstash/certs/cert.pem"
      ...
    }
  }
[2019-02-17T05:19:41,172][ERROR][logstash.agent           ] Cannot create pipeline {:reason=>"Something is wrong with your configuration."}

Please help any one has done TLS/SSL enable at logstash level

Thanks

Please add your complete Elasticsearch output plugin configuration

# File does not exist or cannot be opened /etc/logstash/certs/cert.pem

Does the file exist ? Does the user that logstash process runs under has the necessary permissions to open the file ?

What's the output of

ls -l /etc/logstash/certs/cert.pem

Yes, the file is existed and given 755 permissions

the log stash service is running from root user

Thanks

After fixing few issues now i see below error in logstash log file

[2019-02-26T03:03:24,541][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://logstash_user:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://logstash_user:xxxxxx@localhost:9200/][Manticore::ClientProtocolException] Received fatal alert: bad_certificate"}

any suggestions please to fix this issue

Thanks

Logstash says that it doesn't like the certificate of Elasticsearch. This can happen for a number of reasons, please share the relevant parts of your Logstash configuration, your Elasticsearch configuration and the elasticsearch logs. Otherwise it's really hard to guess what could be going on

Here is the ouput plugin from logstash config

output {
        elasticsearch
        {       hosts => ["hsotname1:9200","hsotname2:9200"]
                user  => "logstash_user"
                password => "XXXXXXXXX"
                index => "indexname"
				ssl => true
                cacert => "/etc/logstash/certs/ca.pem"
        }
}

Below are the parameters i have added in elasticsearch.yml file

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.client_authentication: required

See Security settings in Elasticsearch | Elasticsearch Guide [8.11] | Elastic from which I copy:

Controls the server’s behavior in regard to requesting a certificate from client connections. Valid values are required , optional , and none . required forces a client to present a certificate, while optional requests a client certificate but the client is not required to present one. Defaults to none .

By setting this to required you declare that Logstash (and anything else that connects to Elasticsearch via http ) should authenticate with a client certificate and key. Is this what you want ?
Because your logstash elasticsearch output plugin configuration seems to imply that you want to authenticate with a username and password, and not with a client certificate.

Can you describe what you want to achieve so that we can help you get there?

Hi,

Thank you for your prompt response.

I am trying to enable TLS from log stash to Elastic search. Currently am authenticating with user & password and along with that i would like to add SSL certification as well.

Please suggest.

Thanks

Hi, can you add more details? Do you want to protect the communication with TLS ( this provides integrity and confidentiality to the communication between Logstash and Elasticsearch) or do you also want to authenticate Logstash with a client certificate instead of username/password ? Do you have an explicit requirement for the latter?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.