How to migrate index

Elastic Stack Elasticsearch
1

Hello Guys,
I need to migrate 50 index to a new cluster, I'm going try to explain my situation.

Today I have 1 server with 1 shard, 0 replicas and 50 index, and my new environment is with 1 cluster with 4 ES nodes, 1 shard and 1 replica for each node. So, how can I to move these indexes?, 25 index in one node and 25 index in other node? or 50 indexes in 1 node and Elasticsearch move every index without me?

Thank you.

2

Sorry. But I think you are confused here.

You are talking about 1 server which has 1 shard 0 replica? Only indices have shards and replicas.

So I think you have 1 server which has 50 indices, each index has 1 shard 0 replica. Is that correct?

If you want to add a new server (a new node), just start it (with same cluster name and unicast list of hosts set) and you're done. You will have then 25 shards on one node and 25 shards on the other one.

If you want to have replication, then change live the number of replica for each index. You will end up with 50 shards on one node and 50 shards on the second node. Some of them will be primaries or replicas but you don't have to worry about it.

Makes sense?

3

Hi David, thank you for answered my question.
Yes I have 1 server wihich has 50 indices and each index has 1 shard 0 replica.
I can't add more servers , because I have to migrate to a new environment with a new Vmware cluster. I'm working to create a new ES cluster with 4 ES nodes, with 1 shard 1 replica per node.
If I'll move these 50 indices to new ES cluster and If I'll move all indices to only for 1 node then Elasticsearch will be shared on the 4 nodes automatically? like a load balancer?. What happend with replicas? Elasticsearch shared these replicas automatically?.
Sorry but I don't know yet how to work Elasticsearch with load balancer.

Thank you.

4

Again, this does not make sense in elasticsearch.

So I don't understand what you are looking for. May be others?

5

In every node I have this in the elasticsearch config:
index.number_of_shards: 1
index.number_of_replicas: 1

if I'll move 1 index that index will be have 1 shard and 1 replica?.
If I'll move all indices(50) in some node every index has 1 shard and 1 replica right?.

and after Elasticsearch will be shared on the 4 nodes automatically?.
What happend with replicas? how much time Elasticsearch taked to create every replicas, 50 right?.

Thank you.

6

This means that every index that will be created then will have 1 shard and 1 replica. That does not affect existing indices.

So whatever you "move" it won't change.

7

if I'll move 1 index that index will be have 1 shard and 1 replica?.
If I'll move all indices(50) in some node every index has 1 shard and 1 replica right?.

Depends on what exactly you mean by "move", but probably yes. The number of shards and replicas set in elasticsearch.yml can be overridden when an index is created (and the number of replicas can be changed at any time) so you can't go by the elasticsearch.yml settings alone.

and after Elasticsearch will be shared on the 4 nodes automatically?.

When you add nodes to a cluster all shards will be distributed evenly among the nodes with a few restrictions:

  • Only the number of shards is taken into account, not their size. In other words, you might get different amounts of data on the nodes.
  • The primary shard and its replica will never be placed on the same node.
  • If a node doesn't have sufficient disk space the distribution could be lopsided.
  • Cluster configuration can restrict shard allocation further so that e.g. primary and replica shards won't both be placed in the same rack.

What happend with replicas?

I don't understand this question.

how much time Elasticsearch taked to create every replicas, 50 right?.

Creating an empty replica shard takes no time at all, but the actual time depends on the size of your shards.

8

Hello Magnus,
I got it, Magnus and David thank you very much.