Build-in users' passwords gone after service restart

Elastic Stack Elasticsearch
1

Hi, ELK 7.13.2,

./elasticsearch-setup-passwords interactive

curl -u elastic 'https://localhost:9200/_xpack/security/_authenticate?pretty'

// here authentication as elastic works

systemctl restart elasticsearch

// after service restart, authentication as elastic doesn't work anymore (and the same for other users, including kibana_system for example)
curl -u elastic 'https://localhost:9200/_xpack/security/_authenticate?pretty'

any ideas ?

2

Are you running in docker?

3

not docker, but running in Proxmox container Ubuntu 20.04

4

Do you have persistent storage that survives a restart for your Elasticsearch data path?

5

now it's set to /var/lib/elasticsearch
and yes it should survive normally, at least I don't see why it shouldn't.

6

Please create a new index and verify that it was correctly created. Then restart the service/container (you probably want to test both top verify you will not lose data) and verify it is still there.

7

thanks. but now my one node cluster is broken. actually I was following one article to restore my cluster snapshot and I deleted all indices, that's maybe why now all this happens. so I need first to restore the working state of the cluster...

8

Be careful to not delete system indices as they hold important data.

1 Like
9

According to the docs here Snapshot and restore | Elasticsearch Guide [7.13] | Elastic

You can restore snapshots to a running cluster, which includes all data streams and indices in the snapshot by default.

This means that even system indices must be restored. Because now I'm not sure if I can restore my snapshot after when all indices are deleted.

10

ok found this similar issue Accidentally deleted all indices from the cluster. How do I restore the data? - #7 by TimV

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.