What is the best approach for ELK stack clsuter migration to different location?

Hi Experts,

My current ELK stack(6.3.2) setup is at abc location and now data center is migrating to another xyz location . I have 3 node cluster for ES and one kibana and 1 logstash . Now I have to rebuild the entire setup at the new location . So i will be having brand new servers , separate network etc etc.
My concern here is what will be the best approach? I can think of 2

  1. Snapshot (But because my data is live and indices are per day basis on the old cluster so how I am going to achieve this) I mean without downtime how this is possible ?
  2. Re-index API approach which I think is not possible in my case because i do not have the same network ?

Please suggest what would be the best approach . Waiting for your valuable responses .

Regards
VG

If you have time-based indices where new indices are created at a specific time each day you can try the following approach:

  • Update Logstash to write to both clusters in parallel.
  • Then wait until the indices that were being written to at the time of the change are no longer written to.
  • Take a full snapshot of the old cluster. This will now contain all old data, including the index that was only partially written to the new cluster.
  • Go into the new cluster and delete the partially written indices.
  • Restore all indices that do not exist in the destimation cluster from the snapshot.
  • Check that the new cluster is working and has all data.
  • Change Logstash to only write to the new cluster and remove the old cluster.

If you have time-based indices with longer time periods, e.g. weekly or monthly, or are using the rollover API the process would need to be altered.

This sounds really good approach as i have daily Indices thank you very much @Christian_Dahlqvist . I just need to check if I can dual feed from one LS to 2 clusters as both are on different networks. Also on the old LS server I have syslog which write data to a file then LS read it and index that data , so I need to take care of that as well. Any further suggestion on this please .

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.