Issue moving index between different node/shards configurations

Elastic Stack Elasticsearch
1

I have a question regarding the issue of index copying. I've seen this discussed, but have not found a solution suiting my needs below. I would appreciate any guidance you can provide.

My current situation is such that I have 3 environments (all ES v0.18.4):
(i)process, (ii) prod, (iii) dev

Currently, all 3 have the same configuration:
2 nodes, 5 shards, replication=1

I generally index data on the "process" nodes, then rsync their data directories to the corresponding environment/node where I want to move the data. eg: to move my index from "process" to "dev", I would copy /data from "process node 1" to "dev node 1" and from "process node 2" to "dev node 2".

This works fine as long as every environment has the exact same node/shard config. However, I may soon need 50 nodes in prod, and only 1 in dev. Is there a proper (configuration-independent) method to do this?

To summarize my goal, I want to move data, regardless of X, Y, M, N:
FROM: X process nodes with Y shards
TO: M prod nodes with N shards

2

Then you need to reindex that data, read it from one system and index it
into the other.

On Wed, Mar 28, 2012 at 2:20 AM, rrgrzcool greg.zimerman@gmail.com wrote:

I have a question regarding the issue of index copying. I've seen this
discussed, but have not found a solution suiting my needs below. I would
appreciate any guidance you can provide.

My current situation is such that I have 3 environments (all ES v0.18.4):
(i)process, (ii) prod, (iii) dev

Currently, all 3 have the same configuration:
2 nodes, 5 shards, replication=1

I generally index data on the "process" nodes, then rsync their data
directories to the corresponding environment/node where I want to move the
data. eg: to move my index from "process" to "dev", I would copy /data from
"process node 1" to "dev node 1" and from "process node 2" to "dev node 2".

This works fine as long as every environment has the exact same node/shard
config. However, I may soon need 50 nodes in prod, and only 1 in dev. Is
there a proper (configuration-independent) method to do this?

To summarize my goal, I want to move data, regardless of X, Y, M, N:
FROM: X process nodes with Y shards
TO: M prod nodes with N shards

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/issue-moving-index-between-different-node-shards-configurations-tp3863195p3863195.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.