# Xpack.security.enabled: true - the cluster fails to connect the nodes

**URL:** <https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [September 27, 2019, 2:20pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415 "2019-09-27T14:20:21Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [September 27, 2019, 2:20pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/1 "2019-09-27T14:20:21Z")

</div>

Hi,  
I have setup an elaticsearch cluster of 3 nodes.  
I'm running the latest version of elasticsearch: 7.3.2.  
After setting the xpack.security.enabled to true, the cluster has stopped working.  
After running curl -x to check the health of the cluster, it says that it failed to connect to the node and that the connection is refused.  
Can you help me with this issue ?  
Thank you,

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [September 27, 2019, 2:23pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/2 "2019-09-27T14:23:32Z")

</div>

Please share your configuration and the logs from the elasticsearch nodes. Without this information, it's practically impossible for anyone to help you.

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [September 27, 2019, 2:30pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/3 "2019-09-27T14:30:29Z")

</div>

Hi,

Here is the elasticsearch.yml configuration file:

cluster.name: cluster-elk-prod  
node.name: elk-1 #elk-2 and elk-3 on the other nodes  
path.data: /var/lib/elastic  
path.logs: /var/log/elasticsearch  
network.host: 10.0.1.4  
discovery.seed\_hosts: ["10.0.1.4", "10.0.1.5", "10.0.1.6"]  
cluster.initial\_master\_nodes: ["elk-1"]  
xpack.security.enabled: true

Here are the logs of the cluster:

[1] bootstrap checks failed  
[1]: Transport SSL must be enabled if security is enabled on a [basic] license. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]  
[2019-09-27T14:21:09,806][INFO][o.e.n.Node] [elk-1] stopping ...  
[2019-09-27T14:21:09,838][INFO][o.e.n.Node] [elk-1] stopped  
[2019-09-27T14:21:09,839][INFO][o.e.n.Node] [elk-1] closing ...  
[2019-09-27T14:21:09,855][INFO][o.e.n.Node] [elk-1] closed  
[2019-09-27T14:21:09,857][INFO][o.e.x.m.p.NativeController] [elk-1] Native controller process has stopped - no new native processes can be started

Thank you,

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [September 27, 2019, 2:33pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/5 "2019-09-27T14:33:17Z")

</div>

Results of checking the health of the cluster:  
curl -X GET [http://10.0.1.4:9200/\_cluster/health?pretty](http://10.0.1.4:9200/_cluster/health?pretty)  
curl: (7) Failed to connect to 10.0.1.4 port 9200: Connection refused

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [September 27, 2019, 2:37pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/6 "2019-09-27T14:37:13Z")

</div>

> [@laurentiubanica](#):
>
> [1]: Transport SSL must be enabled if security is enabled on a [basic] license. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]

You need to enable TLS when you use security. See our docs on how to do this : [Configure TLS | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html)

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [September 27, 2019, 2:39pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/7 "2019-09-27T14:39:07Z")

</div>

I have tried enabling TLS, but I don't know how to use the same CA for all the nodes in the cluster.

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [September 27, 2019, 2:49pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/8 "2019-09-27T14:49:55Z")

</div>

Should I generate a single CA and a single certificate to be used by all the nodes ? In this case, how can I transfer these files (CA and certificate) to the other nodes ?

Or should I generate a CA and certificate for every node, separately ?

I couldn't find this info in the documentation.

Thank you,

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [September 27, 2019, 3:30pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/9 "2019-09-27T15:30:05Z")

</div>

Everything that you ask can be found in the docs I shared with you, please read through it again [https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates)

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [September 30, 2019, 3:30pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/11 "2019-09-30T15:30:37Z")

</div>

Please don't post unformatted code, logs, or configuration as it's very hard to read.

Instead, paste the text and format it with \</\> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.

It would be great if you could update your post to solve this.

- Do you _actually need_ to run one Kibana instance per docker container ?
- The docker logs should be much more than this one single line. Please share relevant logs from both kibana and elasticsearch.

The more time you put into providing some very basic and necessary pieces of information up front, the quicker you'll get to be assisted towards a resolution !

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [October 1, 2019, 9:14am UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/12 "2019-10-01T09:14:28Z")

</div>

```
Hi,
I want to run one Kibana docker per instance container for redundancy purpose.

These are couple of logs of the Kibana docker, associated to the authentication event. I can't paste all of them, due to the maximum characters permitted. I don't see any relevant logs coming from the elasticsearch cluster. (/var/log/elasticsearch/cluster-elk-prod.log)

{"type":"response","@timestamp":"2019-10-01T08:59:05Z","tags":[],"pid":6,"method":"get","statusCode":304,"req":{"url":"/bundles/login.bundle.js","method":"get","headers":{"host":"servers","connection":"close","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/login?next=%2F","if-none-match":"\"e0c275e902beb3a821a96713d4988bae15e8bba3-/bundles/-gzip\""},"remoteAddress":"172.17.0.1","userAgent":"172.17.0.1","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/login?next=%2F"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET /bundles/login.bundle.js 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2019-10-01T08:59:06Z","tags":[],"pid":6,"method":"get","statusCode":304,"req":{"url":"/built_assets/dlls/icon.logo_kibana-js.bundle.dll.js","method":"get","headers":{"host":"servers","connection":"close","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/login?next=%2F","if-none-match":"\"7fd622cd3f0956ac15a6ca864b27d30e76123cef-/built_assets/dlls/-gzip\""},"remoteAddress":"172.17.0.1","userAgent":"172.17.0.1","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/login?next=%2F"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /built_assets/dlls/icon.logo_kibana-js.bundle.dll.js 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2019-10-01T08:59:06Z","tags":[],"pid":6,"method":"get","statusCode":304,"req":{"url":"/built_assets/dlls/icon.clock-js.bundle.dll.js","method":"get","headers":{"host":"servers","connection":"close","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0","accept":"*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/login?next=%2F","if-none-match":"\"b01edb462212cd7ba943db47d90dad3cff42e59a-/built_assets/dlls/-gzip\""},"remoteAddress":"172.17.0.1","userAgent":"172.17.0.1","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/login?next=%2F"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /built_assets/dlls/icon.clock-js.bundle.dll.js 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2019-10-01T08:59:06Z","tags":[],"pid":6,"method":"get","statusCode":304,"req":{"url":"/ui/images/bg_top_branded.svg","method":"get","headers":{"host":"servers","connection":"close","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0","accept":"image/webp,*/*","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/built_assets/css/plugins/security/index.light.css","if-modified-since":"Fri, 06 Sep 2019 15:21:47 GMT","if-none-match":"\"db3622756413533cdb3a029b8a6b4e26380bf693-gzip\""},"remoteAddress":"172.17.0.1","userAgent":"172.17.0.1","referer":"https://secops-elk1.westeurope.cloudapp.azure.com/built_assets/css/plugins/security/index.light.css"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /ui/images/bg_top_branded.svg 304 2ms - 9.0B"}
```

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [October 1, 2019, 12:28pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/14 "2019-10-01T12:28:29Z")

</div>

Apologies but I'll have to repeat myself

> [@ikakavas](#):
>
> The docker logs should be much more than this one single line. Please share relevant logs from both kibana and elasticsearch.

There's nothing we can do to help you out with the amount of information you share with us.

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [October 1, 2019, 12:47pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/15 "2019-10-01T12:47:39Z")

</div>

Could please point out the name of the log files? I ran #docker logs kibana and I have looked into /var/log/elasticsearch/cluster.log.

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [October 1, 2019, 1:55pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/16 "2019-10-01T13:55:37Z")

</div>

Start elasticsearch and kibana containers with the configuration that doesn't work. Then run `docker ps` to get the running instances and then run `docker logs <instance_id> --since 10m` on all of them and share the output .If it doesn't fit here, you can use a text hosting service like [https://gist.github.com](https://gist.github.com) or [https://pastebin.com/](https://pastebin.com/)

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [October 2, 2019, 7:43am UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/18 "2019-10-02T07:43:04Z")

</div>

> <https://gist.github.com/laurentiubanica/8a25c670a98a3840559f769fe25e5415>

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [October 2, 2019, 8:36am UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/19 "2019-10-02T08:36:59Z")

</div>

Please provide **all** the information you are asked to.

> [@ikakavas](#):
>
> Start **elasticsearch** and kibana containers

> [@ikakavas](#):
>
> then run `docker logs <instance_id> --since 10m` **on all of them**

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [October 2, 2019, 9:01am UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/20 "2019-10-02T09:01:42Z")

</div>

> <https://gist.github.com/laurentiubanica/ed95c68091d9a13df58ef0a86d7658b3>

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [October 2, 2019, 9:14am UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/22 "2019-10-02T09:14:15Z")

</div>

These are all the logs from the containers.  
Thank you

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [October 2, 2019, 9:26am UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/23 "2019-10-02T09:26:50Z")

</div>

Can't do anything unless you share your elasticsearch logs.

---

<div class="post-metadata">

**Author:** ![laurentiubanica](https://avatars.discourse-cdn.com/v4/letter/l/71e660/32.png) [@laurentiubanica](https://discuss.elastic.co/u/laurentiubanica)\
**Post date:** [October 2, 2019, 1:35pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/25 "2019-10-02T13:35:24Z")

</div>

Hi,  
Thank you for your support.  
I solved the problem. It was coming from the nginx load balancer which was using the Round Robin method, without any directives implemented.

> **[NGINX Docs | HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#method)**
>
> Load balance HTTP traffic across web or application server groups, with several algorithms and advanced features like slow-start and session persistence.

I modified the nginx configuration file with the ip\_hash directive and now Kibana is running with all 3 dockers up&running.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 30, 2019, 1:35pm UTC](https://discuss.elastic.co/t/xpack-security-enabled-true-the-cluster-fails-to-connect-the-nodes/201415/26 "2019-10-30T13:35:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
