Cluster data migration to another system

Elastic Stack Elasticsearch
#1

I have the ES cluster with 9 shards and replicas, where all the data is
stored locally on the nodes. We have the process complete data migration
from one system to another. Since introduction of ES - we need to be able to
migrate also ES indices. We are using Netapp snap-mirror mechanism for the
rest of the data.
What would be the best practice, so, when the second system starts it will
actually build the cluster from migrated data.

Thank you,
Pasha

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Cluster-data-migration-to-another-system-tp4024814.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--

#2

Hi,

If I understand correctly, you can:

  1. Reindex from old cluster to new cluster
  2. Add new nodes to the existing cluster, increase replication factor and,
    after you've verified all data has been replicated over, shut down old
    nodes... slowly, making sure the cluster state doesn't turn red.

Otis

Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html

On Monday, November 5, 2012 1:14:05 AM UTC-5, Pasha wrote:

I have the ES cluster with 9 shards and replicas, where all the data is
stored locally on the nodes. We have the process complete data migration
from one system to another. Since introduction of ES - we need to be able
to
migrate also ES indices. We are using Netapp snap-mirror mechanism for the
rest of the data.
What would be the best practice, so, when the second system starts it will
actually build the cluster from migrated data.

Thank you,
Pasha

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Cluster-data-migration-to-another-system-tp4024814.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--

#3

Hello,

Thank you for your replies and I would like to express myself more clearly (sorry for mess).

  1. We have staging and production system and we have the process in place of move data from production to staging environments
  2. ES cluster is currently configured to write its data on the local cluster machines. I need to store cluster data on the centralized storage (Netapp)
  3. On the staging environment - we perform full cluster restart that points to predefined central storage (Netapp) for this environment and "imports" copied cluster data.
  4. Actually it looks like I need the live replica of the ES cluster data to be saved in central place

So, mu questions are:

  1. What would be the best practice for storing cluster data in centralized place (instead or in parallel with local nodes).
  2. Should I use gateway or point the cluster files to the central storage place?
  3. Are replicas stored with the cluster data upon shutdown and restart?

Thanks,
Pasha

#4