Question on Replica

Elastic Stack Elasticsearch
1

Hi,

I have a Question on replica settings. Would appreciate your help on this.

  1. I was having one node of ES where initially; it was with default setup
    which is 5 shards and 1 replica.
  2. Then I wanted to increase the node to 2.
  3. Before adding another node I stopped the running node. Edited the
    elasticsearch.yml and changed the replica number to 0.
  4. Then I started both the nodes. (Both the nodes has same elasticsearch
    config parameters especially on number of shards and replica)
  5. Now I expect the shards to get distributed among both the nodes.

But the second node replicated entire indices and shards and became more
like a replica.

Is there anything I am doing wrong?

Thanks
Rahul

2
  1. Before adding another node I stopped the running node. Edited the
    elasticsearch.yml and changed the replica number to 0.

The config just acts as a template for new indices. Changing the config
doesn't affect any indices that already exist.

To do that, you should use the index settings API.

clint

3

That was it!!
Thanks a lot Clint. Appreciate your quick response.

On Sun, Aug 5, 2012 at 5:13 PM, Clinton Gormley clint@traveljury.comwrote:

  1. Before adding another node I stopped the running node. Edited the
    elasticsearch.yml and changed the replica number to 0.

The config just acts as a template for new indices. Changing the config
doesn't affect any indices that already exist.

To do that, you should use the index settings API.

Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

4

What I notice is that while re-balancing the nodes are re balancing the
indices not the shards within it?
I was expecting the shards to get rebalanced.
Now Index 1 has all the shards in Node 1, whereas Index 2 has all the 5
shards in Node 2.

Is this an expected behavior?

Thanks
Rahul

On Sun, Aug 5, 2012 at 5:33 PM, Rahul Sharma
rahul.sharma.coder@gmail.comwrote:

That was it!!
Thanks a lot Clint. Appreciate your quick response.

On Sun, Aug 5, 2012 at 5:13 PM, Clinton Gormley clint@traveljury.comwrote:

  1. Before adding another node I stopped the running node. Edited the
    elasticsearch.yml and changed the replica number to 0.

The config just acts as a template for new indices. Changing the config
doesn't affect any indices that already exist.

To do that, you should use the index settings API.

Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

5

Shards are being evened out, index is not really taken into account (and thats the problem you are running into). Thats how ES works now, and you can work a bit around it by shards_per_node setting. It will be improved in the very near future.

On Aug 5, 2012, at 2:20 PM, Rahul Sharma rahul.sharma.coder@gmail.com wrote:

What I notice is that while re-balancing the nodes are re balancing the indices not the shards within it?
I was expecting the shards to get rebalanced.
Now Index 1 has all the shards in Node 1, whereas Index 2 has all the 5 shards in Node 2.

Is this an expected behavior?

Thanks
Rahul

On Sun, Aug 5, 2012 at 5:33 PM, Rahul Sharma rahul.sharma.coder@gmail.com wrote:
That was it!!
Thanks a lot Clint. Appreciate your quick response.

On Sun, Aug 5, 2012 at 5:13 PM, Clinton Gormley clint@traveljury.com wrote:

  1. Before adding another node I stopped the running node. Edited the
    elasticsearch.yml and changed the replica number to 0.

The config just acts as a template for new indices. Changing the config
doesn't affect any indices that already exist.

To do that, you should use the index settings API.
Elasticsearch Platform — Find real-time answers at scale | Elastic

clint

6

Aah I see.
Just FYI, Indexes created afterwards got distributed correctly.

On Wed, Aug 8, 2012 at 3:26 AM, Shay Banon kimchy@gmail.com wrote:

Shards are being evened out, index is not really taken into account (and
thats the problem you are running into). Thats how ES works now, and you
can work a bit around it by shards_per_node setting. It will be improved in
the very near future.

On Aug 5, 2012, at 2:20 PM, Rahul Sharma rahul.sharma.coder@gmail.com
wrote:

What I notice is that while re-balancing the nodes are re balancing the
indices not the shards within it?
I was expecting the shards to get rebalanced.
Now Index 1 has all the shards in Node 1, whereas Index 2 has all the 5
shards in Node 2.

Is this an expected behavior?

Thanks
Rahul

On Sun, Aug 5, 2012 at 5:33 PM, Rahul Sharma <rahul.sharma.coder@gmail.com

wrote:

That was it!!
Thanks a lot Clint. Appreciate your quick response.

On Sun, Aug 5, 2012 at 5:13 PM, Clinton Gormley clint@traveljury.comwrote:

  1. Before adding another node I stopped the running node. Edited the
    elasticsearch.yml and changed the replica number to 0.

The config just acts as a template for new indices. Changing the config
doesn't affect any indices that already exist.

To do that, you should use the index settings API.

Elasticsearch Platform — Find real-time answers at scale | Elastic

clint