Recover Elastic Cluster password

Elastic Stack Elasticsearch
#1

Hello!
For some reason, password/user doesn't work on my cluster, it used to work but now it doesn't. I don't know if it is because I had to reboot all servers (working with 3 servers). Now I would like to reset all password (kibana/elasticsearch/logstash/...). I configured password using this:

bin/elasticsearch-setup-passwords interactive

Probably, I have to run the same command again, but I'm not sure. And want to know if running the command will reset my data on Kibana/Elastic (dashboards/indices).

Also why password is not working? If I restart the server elastic/kibana might forget password?

Thanks

#2 
$ ./bin/elasticsearch-setup-passwords interactive

Failed to determine the health of the cluster running at http://192.168.1.76:9200
Unexpected response code [503] from calling GET http://192.168.1.76:9200/_cluster/health?pretty
Cause: master_not_discovered_exception

It is recommended that you resolve the issues with your cluster before running elasticsearch-setup-passwords.
It is very likely that the password changes will fail when run against an unhealthy cluster.

Do you want to continue with the password setup process [y/N]
#3

You need to look at the logs for your nodes.
Authentication is failing because your cluster is unhealthy, and the index that stores user information cannot be read.

#4

I restarted all servers on cluster and tried to run the command again

bin/elasticsearch-setup-passwords interactive

and got this

root@elasticmaster:/usr/share/elasticsearch/bin# ./elasticsearch-setup-passwords interactive

Failed to authenticate user 'elastic' against http://192.168.1.76:9200/_security/_authenticate?pretty
Possible causes include:
 * The password for the 'elastic' user has already been changed on this cluster
 * Your elasticsearch node is running against a different keystore
   This tool used the keystore at /etc/elasticsearch/elasticsearch.keystore

ERROR: Failed to verify bootstrap password

If setup password once again will I lose all data/indices/dashboard?

#5

That setup tool can only be used once. It is a setup process, not a recovery process. There is no tool that can recover the password for your users.

First you need to work out what happened here.

For some reason, password/user doesn't work on my cluster

That doesn't happen except for a reason. Usually it is a sign that your cluster is temporarily unhealthy and cannot read data from the security index. In more extreme cases it means that it has lost data, and cannot be recovered. No matter what the circumstances there will be a cause and the way you find that out is to look at the logs.
Really. There is no substitute here for going to the logs on each node and finding out why your cluster is not behaving correctly.

1 Like
#6

The problem is: I don't know when it stopped working, maybe days/weeks. So... searching on logs might be too hard. Now, I'm working on remake all p12 certifies. Might this works or should I try other solution? I don't want to lose the data store, I know cluster lose some data, but I want to recover as much as possible.

#7

Which is why you need to check the logs.
Something is wrong with your cluster. If you want to try and recover as much of the data as possible, you need to find out what is going on. Taking recovery steps before you understand the problem is quite likely to make the problem worse, and lead to more data loss.

I'm not going to respond to this thread any further. I have given you the best advice I can - you need to look at your logs.
Alternatively, you can delete everything and start again, but I cannot help you resolve a problem if you refuse to take the recommended steps that would allow us to diagnose it.