# Elastic Kibana TLS/SSL fails on Docker Swarm **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security, docker\ **Created:** [May 22, 2020, 6:03pm UTC](https://discuss.elastic.co/t/elastic-kibana-tls-ssl-fails-on-docker-swarm/233943 "2020-05-22T18:03:45Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Sreekanth3](https://avatars.discourse-cdn.com/v4/letter/s/4491bb/32.png) [@Sreekanth3](https://discuss.elastic.co/u/Sreekanth3)\ **Post date:** [May 22, 2020, 6:03pm UTC](https://discuss.elastic.co/t/elastic-kibana-tls-ssl-fails-on-docker-swarm/233943/1 "2020-05-22T18:03:45Z")
Hi, I have deployed 3 node Elastic & Kibana on a 3 Node Docker Swarm. I could see the Cluster is up & running with 3 node elastic & 1 Kibana. Now, I'm trying to Provide Elasticsearch & Kibana with TLS/SSL authentication. I have generated the certs using below commands: bin/elasticsearch-certutil ca & bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 bin/elasticsearch-certutil cert --ca config/certs/elastic-stack-ca.p12 -name "CN=something,OU=Consulting Team,DC=mydomain,DC=com" openssl pkcs12 -in client.p12 -nocerts -nodes \> client.key openssl pkcs12 -in client.p12 -clcerts -nokeys \> client.cer openssl pkcs12 -in client.p12 -cacerts -nokeys -chain \> client-ca.cer I have copied all these certificates to my local VM & built a docker-compose file as below: > version: "3.7" > services: > es01: > image: [docker.elastic.co/elasticsearch/elasticsearch:7.5.0](http://docker.elastic.co/elasticsearch/elasticsearch:7.5.0) > container\_name: es01 > environment: > - node.name=es01 > - cluster.name=elk-cluster > - discovery.seed\_hosts=es02,es03 > - cluster.initial\_master\_nodes=es01,es02,es03 > - bootstrap.memory\_lock=false > - "ES\_JAVA\_OPTS=-Xms512m -Xmx512m" > - xpack.security.enabled=true > - xpack.security.transport.ssl.enabled=true > - xpack.security.transport.ssl.keystore.type=PKCS12 > - xpack.security.transport.ssl.verification\_mode=certificate > - xpack.security.transport.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - xpack.security.transport.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - xpack.security.transport.ssl.truststore.type=PKCS12 > - xpack.security.http.ssl.enabled=true > - xpack.security.http.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - xpack.security.http.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - xpack.security.http.ssl.client\_authentication=optional > networks: > - dockerelk > ports: > - "9200:9200" > - "9300:9300" > volumes: > - ./elasticsearch/config/certs/elastic-stack-ca.p12:/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - ./elasticsearch/config/certs/elastic-certificates.p12:/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - x2es01:/usr/share/elasticsearch/data > extra\_hosts: > - "host1:192.168.x.x" > - "host2:192.168.x.x" > - "host3:192.168.x.x" > deploy: > replicas: 1 > es02: > image: [docker.elastic.co/elasticsearch/elasticsearch:7.5.0](http://docker.elastic.co/elasticsearch/elasticsearch:7.5.0) > container\_name: es02 > environment: > - node.name=es02 > - cluster.name=elk-cluster > - discovery.seed\_hosts=es01,es03 > - cluster.initial\_master\_nodes=es01,es02,es03 > - bootstrap.memory\_lock=false > - "ES\_JAVA\_OPTS=-Xms512m -Xmx512m" > - xpack.security.enabled=true > - xpack.security.transport.ssl.enabled=true > - xpack.security.transport.ssl.keystore.type=PKCS12 > - xpack.security.transport.ssl.verification\_mode=certificate > - xpack.security.transport.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - xpack.security.transport.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - xpack.security.transport.ssl.truststore.type=PKCS12 > - xpack.security.http.ssl.enabled=true > - xpack.security.http.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - xpack.security.http.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - xpack.security.http.ssl.client\_authentication=optional > networks: > - dockerelk > ports: > - "9201:9200" > - "9301:9300" > volumes: > - ./elasticsearch/config/certs/elastic-stack-ca.p12:/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - ./elasticsearch/config/certs/elastic-certificates.p12:/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - x2es02:/usr/share/elasticsearch/data > extra\_hosts: > - "host1:192.168.x.x" > - "host2:192.168.x.x" > - "host3:192.168.x.x" > deploy: > replicas: 1 > es03: > image: [docker.elastic.co/elasticsearch/elasticsearch:7.5.0](http://docker.elastic.co/elasticsearch/elasticsearch:7.5.0) > container\_name: es03 > environment: > - node.name=es03 > - cluster.name=elk-cluster > - discovery.seed\_hosts=es01,es02 > - cluster.initial\_master\_nodes=es01,es02,es03 > - bootstrap.memory\_lock=false > - "ES\_JAVA\_OPTS=-Xms512m -Xmx512m" > - xpack.security.enabled=true > - xpack.security.transport.ssl.enabled=true > - xpack.security.transport.ssl.keystore.type=PKCS12 > - xpack.security.transport.ssl.verification\_mode=certificate > - xpack.security.transport.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - xpack.security.transport.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - xpack.security.transport.ssl.truststore.type=PKCS12 > - xpack.security.http.ssl.enabled=true > - xpack.security.http.ssl.keystore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - xpack.security.http.ssl.truststore.path=/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - xpack.security.http.ssl.client\_authentication=optional > networks: > - dockerelk > ports: > - "9202:9200" > - "9302:9300" > volumes: > - ./elasticsearch/config/certs/elastic-stack-ca.p12:/usr/share/elasticsearch/config/certs/elastic-stack-ca.p12 > - ./elasticsearch/config/certs/elastic-certificates.p12:/usr/share/elasticsearch/config/certs/elastic-certificates.p12 > - x2es03:/usr/share/elasticsearch/data > extra\_hosts: > - "host1:192.168.x.x" > - "host2:192.168.x.x" > - "host3:192.168.x.x" > deploy: > replicas: 1 kibana01: image: [docker.elastic.co/kibana/kibana:7.5.0](http://docker.elastic.co/kibana/kibana:7.5.0) container\_name: kib01 environment: - ELASTICSEARCH\_URL=[https://192.168.x.x:9200](https://192.168.x.x:9200) - ELASTICSEARCH\_HOSTS=[https://192.168.x.x:9200](https://192.168.x.x:9200) - xpack.security.enabled=true - ELASTICSEARCH\_USERNAME=kibana - ELASTICSEARCH\_PASSWORD=kibana - elasticsearch.ssl.certificate=/usr/share/kibana/config/certs/client.cer - elasticsearch.ssl.key=/usr/share/kibana/config/certs/client.key - elasticsearch.ssl.certificateAuthorities=/usr/share/kibana/config/certs/client-ca.cer - elasticsearch.ssl.verificationMode=certificate networks: - dockerelk ports: - "5601:5601" volumes: - ./kibana/config/certs/client.cer:/usr/share/kibana/config/certs/client.cer - ./kibana/config/certs/client-ca.cer:/usr/share/kibana/config/certs/client-ca.cer - ./kibana/config/certs/client.key:/usr/share/kibana/config/certs/client.key - x2kibgit:/usr/share/kibana networks: dockerelk: external: true volumes: x2es01: x2es02: x2es03: x2kibgit: Also, I have created the dockernet network prior running the compose file. As, I was unable to set passwords using ./elasticsearch-setup-passwords interactive So I created a custom user & password using below command. bin/elasticsearch-users useradd elkuser -p elkpass -r superuser. So, Now I'am able to login lo all my 3 ES instances using [https://192.168.x.x:9200](https://192.168.x.x:9200/) with elkuser & elkpass command. Also, I see my cluster ES is healthy Now. But, not sure why Kibana is not up & running when using TLS/SSL connection. In my Kibana docker container logs I see that following Warnings: {"type":"log","@timestamp":"2020-05-22T14:37:19Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"No living connections"} {"type":"log","@timestamp":"2020-05-22T14:37:20Z","tags":["warning","elasticsearch","data"],"pid":6,"message":"Unable to revive connection: [https://192.168.x.x:9200/](https://192.168.x.x:9200/)"} {"type":"log","@timestamp":"2020-05-22T14:37:20Z","tags":["warning","elasticsearch","data"],"pid":6,"message":"No living connections"} {"type":"log","@timestamp":"2020-05-22T14:37:20Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch for the [data] cluster. Error: No Living connections"} {"type":"log","@timestamp":"2020-05-22T14:37:21Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"Unable to revive connection: [https://192.168.x.x:9200/](https://192.168.x.x:9200/)"} {"type":"log","@timestamp":"2020-05-22T14:37:21Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"No living connections"} {"type":"log","@timestamp":"2020-05-22T14:37:21Z","tags":["warning","elasticsearch","admin"],"pid":6,"message":"Unable to revive connection: [https://192.168.x.x:9200/](https://192.168.x.x:9200/)"} Could someone of you help me in understanding the root cause of this issue & the meaning for these logs.. if either it's related to docker network or swarm setup or it's issue with certificates. Any help would be appreciated. Thanks in Advance ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [May 23, 2020, 10:43am UTC](https://discuss.elastic.co/t/elastic-kibana-tls-ssl-fails-on-docker-swarm/233943/2 "2020-05-23T10:43:35Z")
Please don't open multiple posts for the same issue. I think this is more complete so do you mind deleting [the other one](https://discuss.elastic.co/t/kibana-tls-ssl-fails-on-docker-swarm/233938/2) ? ---
**Author:** ![Sreekanth3](https://avatars.discourse-cdn.com/v4/letter/s/4491bb/32.png) [@Sreekanth3](https://discuss.elastic.co/u/Sreekanth3)\ **Post date:** [May 26, 2020, 4:08pm UTC](https://discuss.elastic.co/t/elastic-kibana-tls-ssl-fails-on-docker-swarm/233943/3 "2020-05-26T16:08:33Z")
yeah , Will remove that old post and could you suggest me anything i should do to get that done @ikakavas. ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [May 28, 2020, 4:14pm UTC](https://discuss.elastic.co/t/elastic-kibana-tls-ssl-fails-on-docker-swarm/233943/4 "2020-05-28T16:14:20Z")
> [@Sreekanth3](#): > > bin/elasticsearch-certutil ca & > bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 > bin/elasticsearch-certutil cert --ca config/certs/elastic-stack-ca.p12 -name "CN=something,OU=Consulting Team,DC=mydomain,DC=com" > openssl pkcs12 -in client.p12 -nocerts -nodes \> client.key > openssl pkcs12 -in client.p12 -clcerts -nokeys \> client.cer > openssl pkcs12 -in client.p12 -cacerts -nokeys -chain \> client-ca.cer These do not contain the output filename that you chose so I'm making assumptions. ```auto - elasticsearch.ssl.certificate=/usr/share/kibana/config/certs/client.cer - elasticsearch.ssl.key=/usr/share/kibana/config/certs/client.key - elasticsearch.ssl.certificateAuthorities=/usr/share/kibana/config/certs/client-ca.cer ``` This means that kibana will attempt to authetnicate itself to elasticsearch using TLS mutual authentication in addition to authenticating with a username and password ```auto - ELASTICSEARCH_USERNAME=kibana - ELASTICSEARCH_PASSWORD=kibana ``` Is this what you want or not? If yes, then you need to enable a PKI realm in elasticsearch If not, you need to remove ```auto - elasticsearch.ssl.certificate=/usr/share/kibana/config/certs/client.cer - elasticsearch.ssl.key=/usr/share/kibana/config/certs/client.key ``` ---
**Author:** ![Sreekanth3](https://avatars.discourse-cdn.com/v4/letter/s/4491bb/32.png) [@Sreekanth3](https://discuss.elastic.co/u/Sreekanth3)\ **Post date:** [May 29, 2020, 12:00pm UTC](https://discuss.elastic.co/t/elastic-kibana-tls-ssl-fails-on-docker-swarm/233943/5 "2020-05-29T12:00:57Z")
Thank you @ikakavas , I will check and update you on this. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 26, 2020, 12:01pm UTC](https://discuss.elastic.co/t/elastic-kibana-tls-ssl-fails-on-docker-swarm/233943/6 "2020-06-26T12:01:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.