How to connect Logstash with Elastic Cloud Elasticsearch via https

[EloaCardoso]

I am having problems to connect my local Logstash with an Elastic Cloud Elasticsearch.

The following lines are placed within the output plugin in the config file:

elasticsearch {
codec => json
manage_template => false
hosts => [ "https://e95444bf74974e44b8c8011d48c96f25.rb-elasticsearch.de.bosch.com:9243" ]
ssl => true
user => "cardoso_fep_user"
password => ""
index => "%{[@metadata][endpoint]}"
document_type => "ObjectEvent"
document_id => "%{epc}"
}

Inside of the logstash.yml I have:

node.name: Logstash6
cloud.id: ":"
cloud.auth: "elastic:"
http.host: "localhost"
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: ["https://e95444bf74974e44b8c8011d48c96f25.rb-elasticsearch.de.bosch.com:9243"]
xpack.monitoring.elasticsearch.username: cardoso_fep_user
xpack.monitoring.elasticsearch.password:

Inside of the Elastic Cloud Enterprise I've created an Elasticsearch and a Kibana node and haven't changed the elasticsearch.yml and kibana.yml files. Both nodes are working. I am able to log in.
This is what I get:

I don't know what I have to do to be able to connect my local Logstash with the Elastic Cloud Elasticsearch...

Does anyone have an idea?

[Dvikas (Vikas)]

Can you provide the configuration of logstash.

[EloaCardoso]

What do you mean? logstash.yml or the .config file?
In the logstash.yml file I've done the changes as I've written above. In the configuration file I've described the elasticsearch output plugin also as above.

[Dvikas (Vikas)]

where u have mentioned the certificate or truststore certificate in logstash.conf file.without that u will get
"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
It means that the certificates are not trusted and does not contain the root certificate for the generated node and client certificates)

[EloaCardoso]

That is a bit complicated. I am using my local laptop just to do some tests with the logstash configuration file. But I don't have access to the certificate, because it is not at my machine.

On the other hand, we have a virtual machine where we've installed docker. Then we've initialized a logstash instance inside of a docker container. In the virtual machine I've placed both files: logstash.yml and the config file. The same files that I have mentioned part of them above. I've mounted both files to the container where I've started logstash.

Do I have to mount the certificate to the logstash container and give the path for the certificate to the field cacert inside of the elasticsearch output plugin? How do I configure the fields inside of elasticsearch output plugin within the logstash config file?

I hope you could understand....

[Dvikas (Vikas)]

output {
elasticsearch {
hosts => ............
user => provide the user
password => provide the pwd
ssl => true
ssl_certificate_verification => true
truststore => "/path/to/truststore.jks"
truststore_password => changeit
}
}
I just provided the sample.Please check and configure that whether is truststore or root_ca .

[EloaCardoso]

Thanks a lot for the sample!

I actually have gotten a .crt file. Then I've done like that:
elasticsearch {
codec => json
manage_template => false
hosts => [ "https://e95444bf74974e44b8c8011d48c96f25.rb-elasticsearch.de.bosch.com:9243" ]
user => "cardoso_fep_user"
password => "mypassword"
index => "%{[@metadata][endpoint]}"
document_type => "ObjectEvent"
document_id => "%{epc}"
ssl => true
ssl_certificate_verification => true
cacert => "pathtoctrfile/crtfile.crt"
}

But it continues with the same problem...
Do you know if there is a way to do that with a crt file?

[EloaCardoso]

I've added the following line in the logstash.yml file:

xpack.monitoring.elasticsearch.ssl.ca: /path/to/ca.crt

and the pipeline doesn't start anymore.

[Dvikas (Vikas)]

error is related to certificate.did u copy the ca.crt file in your logstash folder.I am talking about same ca.crt file which is used in elasticsearch.

[EloaCardoso]

Yes, I've done that.

When I start a docker I do like that:

docker run -d -v /path/logstash/pipeline/:/path/logstash/pipeline/ -v /path/logstash/config/:/path/logstash/config/ -v /pathtoctrlfile/:/pathtoctrlfile/ --name cardoso -p 9600:9600 --net host hostname/elastic.co/logstash:5.6.8

I mount the crt file of my virtual machine to my logtash container. In the config file I give the path of the crt file inside of the container:
cacert => "pathtoctrfile/crtfile.crt"
and in the logstash.yml I've added that:
xpack.monitoring.elasticsearch.ssl.ca: pathtoctrfile/crtfile.crt

[EloaCardoso]

I really don't understand that... I've done exactly like here:
https://www.elastic.co/guide/en/x-pack/5.6/logstash.html#ls-http-ssl
and it doesn't help...
[ERROR][logstash.pipeline] Error registering plugin?
I don't get it