Shards getting relocated after an upgrade

Elastic Stack Elasticsearch
1

Hi all,

I recently upgraded from elasticsearch-0.90.0-1 to elasticsearch-0.90.5.
I've a 2 node cluster with all indices (total of 7) consisting of only
2 shards. So, at all times, for a index, only 1 shards remain on the
node.

After a restart, I expected the cluster to get back to life in few
minutes but the shards started to relocate. That means, the shard that
was earlier present on Node2 started to move to Node1. Isn't this
unnecessary? For now, for a index, one of the shards is relocating and
the other one is unassigned. See the following screenshot to see the
status. http://i.imgur.com/UILwjmA.png

Any reason why its happening? What can I do to avoid this? It takes a
lot of time as it's network IO bound.

Thanks.

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Oh, I think I found the reason. I did the cluster restart by "service
elasticsearch restart" on the 2 nodes.

What I should have done was
Elasticsearch Platform — Find real-time answers at scale | Elastic.

Can someone confirm that this might be the issue?

On Thu, Sep 26, 2013 at 11:58 AM, Abhijeet Rastogi
abhijeet.1989@gmail.com wrote:

Hi all,

I recently upgraded from elasticsearch-0.90.0-1 to elasticsearch-0.90.5.
I've a 2 node cluster with all indices (total of 7) consisting of only
2 shards. So, at all times, for a index, only 1 shards remain on the
node.

After a restart, I expected the cluster to get back to life in few
minutes but the shards started to relocate. That means, the shard that
was earlier present on Node2 started to move to Node1. Isn't this
unnecessary? For now, for a index, one of the shards is relocating and
the other one is unassigned. See the following screenshot to see the
status. http://i.imgur.com/UILwjmA.png

Any reason why its happening? What can I do to avoid this? It takes a
lot of time as it's network IO bound.

Thanks.

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi,
if you do short maintenance on a node, for instance upgrading, you indeed
don't want relocation to happen. You can then temporarily disable
allocation through the cluster settings api:

curl -XPUT localhost:9200/_cluster/settings -d '{
"transient" : {
"cluster.routing.allocation.disable_allocation" : true
}
}'

Don't forget to put it back on once you are done with your maintenance
though:

curl -XPUT localhost:9200/_cluster/settings -d '{
"transient" : {
"cluster.routing.allocation.disable_allocation" : false
}
}'

Also, not related to your problem but it's best practice to issue a flush
right before shutting down each node.

Cheers
Luca

On Thursday, September 26, 2013 8:28:56 AM UTC+2, Abhijeet Rastogi wrote:

Hi all,

I recently upgraded from elasticsearch-0.90.0-1 to elasticsearch-0.90.5.
I've a 2 node cluster with all indices (total of 7) consisting of only
2 shards. So, at all times, for a index, only 1 shards remain on the
node.

After a restart, I expected the cluster to get back to life in few
minutes but the shards started to relocate. That means, the shard that
was earlier present on Node2 started to move to Node1. Isn't this
unnecessary? For now, for a index, one of the shards is relocating and
the other one is unassigned. See the following screenshot to see the
status. http://i.imgur.com/UILwjmA.png

Any reason why its happening? What can I do to avoid this? It takes a
lot of time as it's network IO bound.

Thanks.

--
Regards,
Abhijeet Rastogi (shadyabhi)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.