Copy docs from one index to another index

Elastic Stack Elasticsearch
21

Yeah @dadoonet I used kill command to stop the logstash.

But running logstash with nohup ./bin/logstash -f logstash.conf & will certainly avoid any network connectivity related problems. In my case what happened is Let's say if my old index (es_item) has 20 million docs and tried to copy docs to new index(es_item1), while running in background 20 million docs are copied to new index and again logstash doing copying the same docs with different versions and adding it to new index & docs count is keep on increasing and size of index becomes larger.

In that case only I want to know how to avoid this?

Please let us know your suggestions

Note:- Suppose If i execute logstash without nohup and during copying if any network problem occurs I am getting this error "Network error: Software caused connection abort" and after that logstash copying stops.

Thanks,
Ganeshbabu R

22

Hi @dadoonet,

I want to copy docs with from existing index to new index with filter and based on the filter condition docs should copy to new item index. The filter condition should be like,

  1. In the existing index I have field called as "RGN_ID" based on the RGN_ID the item should copy docs from one index to another index.
  2. Let's say For RGN_id = 9 has the total docs count of 1,03,452 How to add the RGN_ID in logstash conf filter condition?

Please let us know your suggestions and this is required for our workaround testing purpose.

Thanks,
Ganeshbabu R