Elastic search 6.8.1 - missing indices

Good day,

I have an Elasticsearch 6.8.1 single node cluster running inside a kubernetes statefulset to index documents that I need to look after and I noticed that the snapshot path.repo had not been setup previously.

So I added a folder for snapshots with the correct rights on the NFS server where the elastic data currently resides, created a persistent volume bound to the NFS directory I created and changed the statefulset to claim against the persistent volume, and mount it to the container.

I created the repository and started to snapshot the cluster inside the pod - happy days. I've done this for a few of the instances running inside the pods now without a problem.

However, I just ran into into the first situation when I brought the statefulset container back up after adding the path.repo to the config and wanted to start the snapshots, where I have 25 indices normally, I now only have 2 i.e. some indices missing.

Looking on Kibana I can see they are not there, however looking on the NFS data share itself which is bound to a different persistent volume for the data, all the data is still on disk and in the PV.

What can I do to get the indices back?

Prior to setting up the snapshots I stopped all nfs services, pods, and drained the kubernetes node to try and facilitate a state where nothing can touch the location of the cluster data seeing that it is always in a dynamic state.

I then created an atomic disk image using Acronis.

So the question is can I get those back, how to go about it etc. Any help greatly appreciated

Kind Regards

Welcome to our community! :smiley:

We'd definitely encourage you to upgrade. 6.8 reaches EOL very soon, and there's tonnes of good features and improvements in the 7.X series, with 7.15 being the latest version.

What do your Elasticsearch logs show, from startup?