# Configuring x-pack for SSl communication betweeen elasticsearch and logstash **URL:** \ **Category:** Logstash\ **Created:** [June 12, 2017, 7:57am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985 "2017-06-12T07:57:34Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 12, 2017, 7:57am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/1 "2017-06-12T07:57:34Z")
Please help me in using x-pack. My Requirement: My setup is kibana and elasticsearch resides on one machine. Logstash and filebeat resides on another machine. My requirement is to secure the connection with SSL certificates between logstash and elasticsearch. Steps i followed: (1) I have installed x-pack on ElasticSearch.Using default username/password (elastic/changeme). (2) I have self signed certificates generated for client and server. In elasticsearch.yml, i made the following changes, ```auto xpack.ssl.key: "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/ELK_SSL_Certificates/server/pkcs8.key" xpack.ssl.certificate: "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/ELK_SSL_Certificates/server/cert.pem" xpack.ssl.certificate_authorities: ["D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/ELK_SSL_Certificates/testca/cacert.pem"] ``` (3) I am able to successfully establish SSL connection established between filebeat and logstash, Below is filebeat.yml and logstash.conf changes. **filebeat.yml:** ```auto output.logstash: #The Logstash hosts hosts: ["localhost:5043"] ssl.enabled: true ssl.certificate_authorities: ["D:/Softwares/ELK/ELK_SSL_Certificates/testca/cacert.pem"] ssl.certificate: "D:/Softwares/ELK/ELK_SSL_Certificates/client/cert.pem" ssl.key: "D:/Softwares/ELK/ELK_SSL_Certificates/client/pkcs8.key" ``` **logstash.conf:** ```auto input { beats { port => "5043" ssl => true ssl_certificate_authorities => "D:/Softwares/ELK/ELK_SSL_Certificates/testca/cacert.pem" ssl_certificate => "D:/Softwares/ELK/ELK_SSL_Certificates/server/cert.pem" ssl_key => "D:/Softwares/ELK/ELK_SSL_Certificates/server/pkcs8.key" ssl_verify_mode => "force_peer" } } filter{ grok { match => {"message" =>"%{TIMESTAMP_ISO8601:timestamp} %{GREEDYDATA:keyval}"} } kv { source => "keyval" field_split => "," #trimkey => "\s" remove_field => ["keyval"] } } output { elasticsearch { hosts => ["localhost:9200"] user => "elastic" password => "changeme" } stdout { codec => rubydebug } } ``` (4) now i want to secure the connection with SSL between elasticsearch and logstash in the same way i was able to secure connection between filebeat and logstash. I have not installed x-pack in logstash yet. Please let me know on how to achieve it. Am i on right track? ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 13, 2017, 4:35pm UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/2 "2017-06-13T16:35:28Z")
can anyone please help me ---
**Author:** ![joshbressers](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joshbressers/32/42332_2.png) [@joshbressers](https://discuss.elastic.co/u/joshbressers)\ **Post date:** [June 13, 2017, 4:55pm UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/3 "2017-06-13T16:55:11Z")
This link to the x-pack documentation should be what you're looking for [https://www.elastic.co/guide/en/x-pack/current/logstash.html#ls-http-ssl](https://www.elastic.co/guide/en/x-pack/current/logstash.html#ls-http-ssl) ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 14, 2017, 5:53am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/4 "2017-06-14T05:53:23Z")
Hi @joshbressers , Thanks for the link. I added the below lines in logstash.config file, output { elasticsearch { hosts =\> ["localhost:9200"] user =\> "elastic" password =\> "changeme" ssl =\> true cacert =\> ["D://Softwares//ELK//ELK\_SSL\_Certificates//testca//cacert.pem"] } stdout { codec =\> rubydebug } } after adding ssl lines as mentioned in the link provided, i get below error when i start logstash , **11:21:49.029 [[main]-pipeline-manager] ERROR logstash.agent - Pipeline aborted due to error {:exception=\>#\** Please tell me is there any thing else i am missing here ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 14, 2017, 6:06am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/5 "2017-06-14T06:06:32Z")
@magnusbaeck .. Please give me your inputs here. ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [June 14, 2017, 6:50am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/6 "2017-06-14T06:50:48Z")
> [@vinod\_hy](#): > > 11:21:49.029 [[main]-pipeline-manager] ERROR logstash.agent - Pipeline aborted due to error {:exception=\># Please post the error message **in full** , and include in a "code block" in your message by putting 3 backticks before and after the message like this: ``` 11:21:49.029 [[main]-pipeline-manager] ERROR logstash.agent - Pipeline aborted due to error {:exception=\># ``` ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 14, 2017, 6:57am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/7 "2017-06-14T06:57:30Z")
@TimV '''ERROR logstash.agent - Pipeline aborted due to error {:exception=\>#\ **Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [June 14, 2017, 7:01am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/8 "2017-06-14T07:01:12Z") It doesn't look like you have enabled SSL inside elasticsearch. You need to explicitly [turn on SSL](https://www.elastic.co/guide/en/x-pack/current/security-settings.html#http-tls-ssl-settings) for the http (REST) interface. ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 14, 2017, 9:20am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/9 "2017-06-14T09:20:56Z")
@TimV, Below are the changes i have made in my elasticsearch.yml, xpack.security.http.ssl.enabled: true xpack.security.http.ssl.client\_authentication: required xpack.ssl.key: "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/ELK\_SSL\_Certificates/server/pkcs8.key" xpack.ssl.certificate: "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/ELK\_SSL\_Certificates/server/cert.pem" xpack.ssl.certificate\_authorities: "D:/Softwares/ELK/elasticsearch-5.4.0/elasticsearch-5.4.0/config/x-pack/ELK\_SSL\_Certificates/testca/cacert.pem" Am i missing any thing else here. ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 14, 2017, 9:57am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/10 "2017-06-14T09:57:29Z")
@TimV Hello Tim, I am able to establish https connection to elasticsearch. In browser, i could see this ![](https://us1.discourse-cdn.com/elastic/original/3X/f/0/f057952b33ef59321f6968da51386fa75cb64f44.png) But in elasticsearch.log, i get , ''' **io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:**''' ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [June 14, 2017, 11:41am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/11 "2017-06-14T11:41:27Z")
> [@vinod\_hy](#): > > '''io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:''' That means that something tried to connect to your ES cluster using HTTP rather than HTTPS. Without more information, I can't tell you where that connection came from. ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 14, 2017, 12:09pm UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/12 "2017-06-14T12:09:08Z")
@TimV Hello Tim, The exception is no more seen. I tried accessing to elasticsearch using http a few times. Thats the reason it threw that exception. I did not realise. Apologise for the same. Now the next issue is i am unable to connect to elasticsearch from logstash and kibana. below is my logstash.conf output section, output { elasticsearch { hosts =\> ["localhost:9200"] user =\> "elastic" password =\> "changeme" ssl =\> true cacert =\> ["D://Softwares//ELK//ELK\_SSL\_Certificates//testca//cacert.pem"] } stdout { codec =\> rubydebug } } I get below error, '''**:error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[https://elastic:xxxxxx@localhost:9200/](https://elastic:xxxxxx@localhost:9200/)][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}**''' ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 15, 2017, 3:36am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/13 "2017-06-15T03:36:50Z")
@TimV Hi Tim, My elasticsearch is also up and running with credentials elastic/changeme. ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [June 15, 2017, 4:01am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/14 "2017-06-15T04:01:08Z")
> [@](#): > > sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target This indicates that you have a problem with the CA chain for your ES certificate. My best guess is this: > [@](#): > > cacert =\> ["D://Softwares//ELK//ELK\_SSL\_Certificates//testca//cacert.pem"] I'm not a logstash expert, but I don't believe that the cacert option to the logstash-elasticsearch-output plugin allows arrays. Can you try: ```auto cacert => "D:/Softwares/ELK/ELK_SSL_Certificates/testca/cacert.pem" ``` ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 15, 2017, 4:15am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/15 "2017-06-15T04:15:16Z")
@TimV i tried that also.. Still same issue. For your information, i have installed x-pack in elasticsearch and made changes in elasticsearch.yml as shared above. I have also installed x-pack in logstash and had made below changes, **In logstash.yml,** xpack.monitoring.elasticsearch.url: ["[https://132.186.102.39:9200](https://132.186.102.39:9200)"] xpack.monitoring.elasticsearch.username: elastic xpack.monitoring.elasticsearch.password: changeme Now i **commented** those lines also. The logstash.yml is a fresh file now with all lines commented. Please note that i have added below line in elasticsearch.yml, **network.host: 132.186.102.39** now i see logstash trying to connect to elasticsearch using **logstash\_system** and **elastic** as username. Please find the logs below, '''10:39:24.433 [Ruby-0-Thread-7: D:/Softwares/ELK/logstash-5.4.0/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:234] INFO logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[http://logstash\_system:xxxxxx@localhost:9200/](http://logstash_system:xxxxxx@localhost:9200/), :path=\>"/"} 10:39:24.437 [Ruby-0-Thread-10: D:/Softwares/ELK/logstash-5.4.0/logstash-5.4.0/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline 10:39:24.454 [Ruby-0-Thread-6: D:/Softwares/ELK/logstash-5.4.0/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:234] WARN logstash.outputs.elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\, :error\_type=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[http://logstash\_system:xxxxxx@localhost](http://logstash_system:xxxxxx@localhost): 9200/][Manticore::SocketException] Connection refused: connect"}''' '''10:39:24.612 [Ruby-0-Thread-12: D:/Softwares/ELK/logstash-5.4.0/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:234] INF O logstash.outputs.elasticsearch - Running health check to see if an Elasticsearch connection is working {:healthcheck\_url=\>[https://elastic:xxxxxx@132.186.102.39:9200/](https://elastic:xxxxxx@132.186.102.39:9200/), :path=\>"/"} 10:39:24.647 [Ruby-0-Thread-12: D:/Softwares/ELK/logstash-5.4.0/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-6.3.0-java/lib/logstash/outputs/elasticsearch/http\_client/pool.rb:234] WAR N logstash.outputs.elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error. {:url=\>#\, :error\_type=\>LogStash: :Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=\>"Elasticsearch Unreachable: [[https://elastic:xxxxxx@132.186.102.39:9200/](https://elastic:xxxxxx@132.186.102.39:9200/)][Manticore::ClientProtocolException] KeyUsage does not allow digit al signatures"}''' Strange thing is that when it uses **logstash\_system** , it uses **localhost** instead of **ip adress**. When it uses **elastic** , it uses **ip\_address** instead of **localhost**. ---
**Author:** ![vinod\_hy](https://avatars.discourse-cdn.com/v4/letter/v/c4cdca/32.png) [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Post date:** [June 15, 2017, 6:04am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/16 "2017-06-15T06:04:22Z")
Can anyone from logstash team help me here . ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 13, 2017, 6:04am UTC](https://discuss.elastic.co/t/configuring-x-pack-for-ssl-communication-betweeen-elasticsearch-and-logstash/88985/17 "2017-07-13T06:04:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.