How can I move/migrate part of my data to a new cluster?

Elastic Stack Elasticsearch
1

Hi,

I have an ES cluster that runs on version 6.8.23 and I have a new cluster that runs version 7.17.1. I want to migrate data not older than 1 year to my new cluster. Is this possible? How can I do it?

2

If you have a timestamp in your index names, you can snapshot only the right indices and restore them in the new cluster.

If not, and if you have a timestamp in your documents, you can use the reindex from remote API with a query.

1 Like
3

there is another method you can use is run logstash
input = Elasticsearch (old)
output = Elasticsearch(new)

I did this in past, don't remember what was the reason. but it works.

1 Like
4

How can I take a snapshot of my index using the time stamp?

I copied (export) the objects (dashboard, index patterns, etc) from the old cluster to the new cluster but not the indices. We are going to use the same pipelines too but I dont have the indices and I dont know how to do copy the mapping. I just need a small snapshot of the indices ( crossing fingers).

5

If the index name contains a date, you can run something like:

PUT /_snapshot/my_repository/snapshot_2?wait_for_completion=true
{
  "indices": "index-date1,index-date2",
  "include_global_state": false
}

See Create snapshot API | Elasticsearch Guide [8.11] | Elastic

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.