Logstash to elasticsearch

Elastic Stack Logstash
1

I think logstash is having trouble sending data to elasticsearch.

To secure beats communications, I've set elasticsearch to use https. I think because of that logstash is having some trouble sending data to elasticsearch.

I get something like this from logstash:

[WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://logstash_internal:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logstash_internal:xxxxxx@localhost:9200/][Manticore::ClientProtocolException] localhost:9200 failed to respond"}

this is my logstash .conf:

output{
elasticsearch {
hosts => ["localhost:9200"]
index => "cleandata"
user => "logstash_internal"
password => "password"
} }

this is the end of my elasticsearch.yml:

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: "http.p12"

I assume that in my logstash .conf within that output {}, I need to specify some additional context but I'm not sure what and in what syntax. I'm scratching my head around it and feel a little lost. Does anyone know what's missing? Thanks ahead!

2

I take it that you mean elasticsearch expects clients to connect using TLS. In that case you need to configure the elasticsearch output to use TLS, which you have not done...

This blog has the step-by-step for configuring TLS between logstash and elasticsearch.

3

Thanks for the response,

I tried adding https and I got a similar error:

[WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://logstash_internal:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://logstash_internal:xxxxxx@localhost:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
4

Looks like you did not correctly configure cacert on the elasticsearch output.

1 Like
5

Thanks for the response,

I seem to be missing the "cacert" then. I know this is astromonically dumb of me but i have got so many of certs I'm not sure which one I need to give it. Would the path for the right cert be in the elasticsearch.yml?

6

You can extract the CA cert from the certificate that you have configured elasticsearch to use.

If you are on Windows, then follow these instruction.

On UNIX you would use openssl.

1 Like
7

Thanks!!

Had to do: openssl pkcs12 -in elastic-stack-ca.p12 -clcerts -nokeys -out http.crt (enter for blank password)

Then in logstash's .conf: direct it to the new file: cacert => "/etc/elk/http.crt" (make sure elastic has permission to the path and the file itself)

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.