We are trying to move Elasticsearch Data from Docker Container to Kubernetes Pod , the default Data path for ES pod is /usr/share/elasticsearch/data/nodes/0/

[AnushaK]

Hi ,

I am trying to copy the indices from the source Elasticsearch to the target Elasticsearch under /usr/share/Elasticsearch/data/nodes/0/indices/. Source and target path is same /usr/share/Elasticsearch/data/nodes/0/indices/ .
But some how the indices are not loaded in the Kibana Front End.

Not getting any error message as well.
Can you please help here.

Steps followed :

docker cp es-container-name:/usr/share/Elasticsearch/data /backup-folder
Copied this backup folder to local machine
kubectl cp -r data/indices/ podname:/usr/share/Elasticsearch/data/nodes/0/indices/

indices is being copied but unable to view those indices in Kibana Index or index patterns

[Tomo_M (Tomohiro Mitani)]

Taking a snapshot is the only reliable and supported way to back up a cluster. You cannot back up an Elasticsearch cluster by making copies of the data directories of its nodes. There are no supported methods to restore any data from a filesystem-level backup. If you try to restore a cluster from such a backup, it may fail with reports of corruption or missing files or other data inconsistencies, or it may appear to have succeeded having silently lost some of your data.

A copy of the data directories of a cluster’s nodes does not work as a backup because it is not a consistent representation of their contents at a single point in time. You cannot fix this by shutting down nodes while making the copies, nor by taking atomic filesystem-level snapshots, because Elasticsearch has consistency requirements that span the whole cluster. You must use the built-in snapshot functionality for cluster backups.

Snapshot and restore | Elasticsearch Guide [8.11] | Elastic

According to this document, recommended method to transfer indices is making snapshot on some repository and loading them to other ES cluster as read-only repository and restore the desired index from the snapshot.

Another way is reindex from a remote cluster.

[Nisha_Shaik]

Hi @Tomo_M

Thank you for the response .

The Current Scenario is We are taking a backup of ES cluster which has been deployed as Docker Container on a VM . We need to restore it to a pod on Kubernetes cluster which is deployed on cloud environment
How will snapshot & restore scenario will work ? Is there any way that can help us to backup & restore the data of above mentioned scenario.

Thanks
Nisha

[Christian_Dahlqvist]

File system snapshots are not supported (see the link provided earlier), so start up the node on docker and create a proper snapshot on shared storage so that you then can restore it to the new node.