# ES cluster break after install x-pack **URL:** \ **Category:** Elasticsearch\ **Created:** [June 12, 2018, 10:11am UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509 "2018-06-12T10:11:46Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\ **Post date:** [June 12, 2018, 10:11am UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/1 "2018-06-12T10:11:47Z")
I have created ES cluster on MAC laptop with 1 client, 3 master and 3 data nodes. ```auto $ cat multinode_es.sh #!/bin/bash ### Client node $HOME/Documents/ELK/ELASTIC/clientnode/elasticsearch-6.2.4/bin/elasticsearch & ### Master nodes $HOME/Documents/ELK/ELASTIC/masternode/node1/elasticsearch-6.2.4/bin/elasticsearch & $HOME/Documents/ELK/ELASTIC/masternode/node2/elasticsearch-6.2.4/bin/elasticsearch & $HOME/Documents/ELK/ELASTIC/masternode/node3/elasticsearch-6.2.4/bin/elasticsearch & ### Data nodes $HOME/Documents/ELK/ELASTIC/datanode/node1/elasticsearch-6.2.4/bin/elasticsearch & $HOME/Documents/ELK/ELASTIC/datanode/node2/elasticsearch-6.2.4/bin/elasticsearch & $HOME/Documents/ELK/ELASTIC/datanode/node3/elasticsearch-6.2.4/bin/elasticsearch & ``` Everything was working fine with this setup before installing the x-pack, i have installed the x-pack on all master nodes using elasticsearch-plugin command. Killed all the ES related processes and started them again but after that it's getting failed with the following error: ---
**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\ **Post date:** [June 12, 2018, 10:12am UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/2 "2018-06-12T10:12:44Z")
[https://paste.fedoraproject.org/paste/0h~v7KkZhXZdj~XFkTtCwQ](https://paste.fedoraproject.org/paste/0h~v7KkZhXZdj~XFkTtCwQ) Since cluster is not started properly hence x-pack command to generate the password is also not working. ```auto $ bin/x-pack/setup-passwords auto Connection failure to: http://127.0.0.1:9200/_xpack/security/_authenticate?pretty failed: Connection refused (Connection refused) ERROR: Failed to connect to elasticsearch at http://127.0.0.1:9200/_xpack/security/_authenticate?pretty. Is the URL correct and elasticsearch running? ``` Strangely my head plugin which is installed as chrome extension is asking for the password to connect with ES before installing x-pack it was not asking for password. Why ES is not starting properly and which password I am supposed to put in head so that it can fetch the ES information? ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [June 12, 2018, 10:32am UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/3 "2018-06-12T10:32:50Z")
> [@Vikrant\_Aggarwal](#): > > i have installed the x-pack on all master nodes You need to install X-Pack on **all** your nodes, not only on the master nodes. ---
**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\ **Post date:** [June 12, 2018, 12:44pm UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/4 "2018-06-12T12:44:06Z")
Thanks it helped me to start the cluster but while resetting the password I am getting this error. ```auto $ bin/x-pack/setup-passwords interactive Failed to authenticate user 'elastic' against http://127.0.0.1:9200/_xpack/security/_authenticate?pretty Possible causes include: * The password for the 'elastic' user has already been changed on this cluster * Your elasticsearch node is running against a different keystore This tool used the keystore at /Users/viaggarw/Documents/ELK/ELASTIC/datanode/node3/elasticsearch-6.2.4/config/elasticsearch.keystore ERROR: Failed to verify bootstrap password HAM-VIAGGARW-02:elasticsearch-6.2.4 viaggarw$ cat /Users/viaggarw/Documents/ELK/ELASTIC/datanode/node3/elasticsearch-6.2.4/config/elasticsearch.keystore ``` I am not able to read the content of mentioned file. I have tried to login using elastic/changeme credentials as indicated in other discussions but that didn't work ```auto $ curl -u elastic -XGET 'http://localhost:9200/_cat/health?v' Enter host password for user 'elastic': {"error":{"root_cause":[{"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401} ``` ---
**Author:** ![deepybee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deepybee/32/20381_2.png) [@deepybee](https://discuss.elastic.co/u/deepybee)\ **Post date:** [June 12, 2018, 1:17pm UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/5 "2018-06-12T13:17:18Z")
> [@Vikrant\_Aggarwal](#): > > Strangely my head plugin which is installed as chrome extension is asking for the password to connect with ES before installing x-pack it was not asking for password. > > Why ES is not starting properly and which password I am supposed to put in head so that it can fetch the ES information? Because by installing X-Pack with the defaults you have secured the REST API that your plugin previously spoke to unsecured. > [@](#): > > I am not able to read the content of mentioned file. This is by design, it's an encrypted Java keystore meant to protect secrets. You can check which keys (but not their passwords) the keystore contains by entering `bin/elasticsearch-keystore list` in the path for that node (note it is _not_ the `x-pack` directory within `bin/`. As this is on your laptop, your easiest option is probably to stop your cluster, remove X-Pack from all 7 nodes with `bin/elasticsearch-plugin remove x-pack` double check a keystore is not present in any config path, then reinstall to _all_ nodes with `bin/elasticsearch-plugin install x-pack` then start the cluster back up and run `bin/x-pack/setup-passwords interactive` ---
**Author:** ![deepybee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deepybee/32/20381_2.png) [@deepybee](https://discuss.elastic.co/u/deepybee)\ **Post date:** [June 12, 2018, 1:23pm UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/6 "2018-06-12T13:23:45Z")
The documentation for bootstrap passwords when installing X-Pack can be found [here](https://www.elastic.co/guide/en/x-pack/6.2/setting-up-authentication.html#bootstrap-elastic-passwords). ---
**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\ **Post date:** [June 12, 2018, 1:38pm UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/7 "2018-06-12T13:38:58Z")
Thanks for your reply. Removed the xpack using --purge option. ```auto $ sh multinode_remove_xpack.sh -> removing [x-pack]... -> removing [x-pack]... -> removing [x-pack]... -> removing [x-pack]... -> removing [x-pack]... -> removing [x-pack]... -> removing [x-pack]... ``` This is the keystore present in keystore. I deleted from one of the node, should I delete it from all nodes? ```auto $ sh multinode_keystore.sh keystore.seed keystore.seed keystore.seed keystore.seed keystore.seed keystore.seed ``` ---
**Author:** ![deepybee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deepybee/32/20381_2.png) [@deepybee](https://discuss.elastic.co/u/deepybee)\ **Post date:** [June 12, 2018, 2:27pm UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/8 "2018-06-12T14:27:15Z")
Yes I would say that is your best option, you want a clean install of X-Pack to all nodes, not one where they pick up an existing keystore and lead you into the same problem. ---
**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\ **Post date:** [June 12, 2018, 3:31pm UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/9 "2018-06-12T15:31:20Z")
I cleared the keystores after the uninstallation of x-pack. Started the ES nodes and verified that I am able to access the setup without credentials. Again did the installation of x-pack, stopped/started the ES services. it's asking for credentials. ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [June 12, 2018, 8:45pm UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/10 "2018-06-12T20:45:21Z")
You need to run ```auto bin/x-pack/setup-passwords interactive ``` after you start the cluster to set the credentials for the reserved users.. ---
**Author:** ![Vikrant\_Aggarwal](https://avatars.discourse-cdn.com/v4/letter/v/f14d63/32.png) [@Vikrant\_Aggarwal](https://discuss.elastic.co/u/Vikrant_Aggarwal)\ **Post date:** [June 13, 2018, 3:26am UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/11 "2018-06-13T03:26:05Z")
Thanks I was able to change this time. but it should have some default password for the configured users because as soon as you install x-pack it stated asking for the password. changeme doesn't work while trying to authenticating the api. ```auto $ bin/x-pack/setup-passwords interactive Initiating the setup of passwords for reserved users elastic,kibana,logstash_system. You will be prompted to enter passwords as the process progresses. Please confirm that you would like to continue [y/N]y Enter password for [elastic]: Reenter password for [elastic]: Enter password for [kibana]: Reenter password for [kibana]: Enter password for [logstash_system]: Reenter password for [logstash_system]: Changed password for user [kibana] Changed password for user [logstash_system] Changed password for user [elastic] ``` ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [June 13, 2018, 6:41am UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/12 "2018-06-13T06:41:05Z")
> [@Vikrant\_Aggarwal](#): > > but it should have some default password for the configured users Hi. No it shouldn't. Removing the default `changeme` password was a deliberate change in 6.0 with the user's security in mind while at the same time preserving the ease of installation. You can read more about it in [this blog post](https://www.elastic.co/blog/default-password-removal-elasticsearch-and-x-pack-6-0) > [@Vikrant\_Aggarwal](#): > > as soon as you install x-pack it stated asking for the password. If by "it" you refer to the elasticsearh head chrome extention, I won't comment on a 3rd party software but our documentation on running `setup-passwords` right after you start your cluster for the first time after installing XPack are, I believe, [clear](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/installing-xpack-es.html). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 11, 2018, 6:41am UTC](https://discuss.elastic.co/t/es-cluster-break-after-install-x-pack/135509/13 "2018-07-11T06:41:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.