How can I move/migrate part of my data to a new cluster?


I have an ES cluster that runs on version 6.8.23 and I have a new cluster that runs version 7.17.1. I want to migrate data not older than 1 year to my new cluster. Is this possible? How can I do it?

If you have a timestamp in your index names, you can snapshot only the right indices and restore them in the new cluster.

If not, and if you have a timestamp in your documents, you can use the reindex from remote API with a query.

1 Like

there is another method you can use is run logstash
input = Elasticsearch (old)
output = Elasticsearch(new)

I did this in past, don't remember what was the reason. but it works.

1 Like

How can I take a snapshot of my index using the time stamp?

I copied (export) the objects (dashboard, index patterns, etc) from the old cluster to the new cluster but not the indices. We are going to use the same pipelines too but I dont have the indices and I dont know how to do copy the mapping. I just need a small snapshot of the indices ( crossing fingers).

If the index name contains a date, you can run something like:

PUT /_snapshot/my_repository/snapshot_2?wait_for_completion=true
  "indices": "index-date1,index-date2",
  "include_global_state": false

See Create snapshot API | Elasticsearch Guide [8.2] | Elastic