Hi Experts,
My current ELK stack(6.3.2) setup is at abc location and now data center is migrating to another xyz location . I have 3 node cluster for ES and one kibana and 1 logstash . Now I have to rebuild the entire setup at the new location . So i will be having brand new servers , separate network etc etc.
My concern here is what will be the best approach? I can think of 2
- Snapshot (But because my data is live and indices are per day basis on the old cluster so how I am going to achieve this) I mean without downtime how this is possible ?
- Re-index API approach which I think is not possible in my case because i do not have the same network ?
Please suggest what would be the best approach . Waiting for your valuable responses .
Regards
VG
If you have time-based indices where new indices are created at a specific time each day you can try the following approach:
- Update Logstash to write to both clusters in parallel.
- Then wait until the indices that were being written to at the time of the change are no longer written to.
- Take a full snapshot of the old cluster. This will now contain all old data, including the index that was only partially written to the new cluster.
- Go into the new cluster and delete the partially written indices.
- Restore all indices that do not exist in the destimation cluster from the snapshot.
- Check that the new cluster is working and has all data.
- Change Logstash to only write to the new cluster and remove the old cluster.
If you have time-based indices with longer time periods, e.g. weekly or monthly, or are using the rollover API the process would need to be altered.
This sounds really good approach as i have daily Indices thank you very much @Christian_Dahlqvist . I just need to check if I can dual feed from one LS to 2 clusters as both are on different networks. Also on the old LS server I have syslog which write data to a file then LS read it and index that data , so I need to take care of that as well. Any further suggestion on this please .