I have a question regarding the issue of index copying. I've seen this discussed, but have not found a solution suiting my needs below. I would appreciate any guidance you can provide.
My current situation is such that I have 3 environments (all ES v0.18.4):
(i)process, (ii) prod, (iii) dev
Currently, all 3 have the same configuration:
2 nodes, 5 shards, replication=1
I generally index data on the "process" nodes, then rsync their data directories to the corresponding environment/node where I want to move the data. eg: to move my index from "process" to "dev", I would copy /data from "process node 1" to "dev node 1" and from "process node 2" to "dev node 2".
This works fine as long as every environment has the exact same node/shard config. However, I may soon need 50 nodes in prod, and only 1 in dev. Is there a proper (configuration-independent) method to do this?
To summarize my goal, I want to move data, regardless of X, Y, M, N:
FROM: X process nodes with Y shards
TO: M prod nodes with N shards