As of now old cluster in use and by the time of new cluster use, we see some new documents were not synced as apps are still using the old cluster. How to avoid these sync issues in cluster migration.
I would decide on a date for switching from old to new cluster.
I would make sure anything that is indexing to old cluster is paused. Then either use re-index API or snapshot to migrate data from old to new cluster.
Then make sure your application and indexer are pointing to new cluster.
This to be can be the best way to ensure you dont lose your data and have smooth migration.
You can achieve a much shorter pause by using cross-cluster replication to replicate documents from the old cluster to the new. That way you can do this:
halt the indexing traffic on the old cluster
wait for CCR to finish copying the last few documents
@mouli_v The other thing I would suggest is to start doing snapshot restore even before your maintenance window. Snapshots are incremental. So while your first snapshot and restore will take some time, subsequent ones will be a lot faster.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.