Suggest? Move (Not copy) large volume (around 100m) data from one index to another

Elastic Stack Elasticsearch
1

One Time Requirement to move unwanted data (around 200mi) from a large (500mi) index based on certain filter criteria to another (New) index.
We are planning to write java code as below.

  1. Group certain list of ids and run "reindex" command in loops to copy to "NEW" index
  2. Group the same list of ids and run "Delete" command in current index

Looking for any "MOVE" kind of solution rather than Copy & Delete approach?

Thank You for any Suggestion.

2

There isn't one. The approach you described is what you need to do.

The only alternative I can think of would be to first clone the index and then delete the appropriate data from the two indices, but as you are moving less than half of the data that may be slower.

2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.