# Inefficient and unnecassary rebalancing when re-opening a closed index **URL:** \ **Category:** Elasticsearch\ **Created:** [December 4, 2012, 7:03pm UTC](https://discuss.elastic.co/t/inefficient-and-unnecassary-rebalancing-when-re-opening-a-closed-index/9937 "2012-12-04T19:03:38Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [December 4, 2012, 7:03pm UTC](https://discuss.elastic.co/t/inefficient-and-unnecassary-rebalancing-when-re-opening-a-closed-index/9937/1 "2012-12-04T19:03:38Z")
When re-opening a closed index, the cluster will move around the shards even though the allocation should be even. Most recent example: re-opening a 4-shard/1-replica index on an 8 node cluster. Each node had before closing the index exactly 1 shard. After re-opening, 4 of the shards remained where they were, but the other 4 moved to recovering on different nodes. The primary delegation for some shards also changed after re-opening, so it is not just that the replicas are recovering. Moving around just 1 replica of an index is a lot of network traffic. Is there any way to avoid this situation? Cheers, Ivan -- ---
**Author:** ![Dan\_Lecocq](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_lecocq/32/2565_2.png) [@Dan\_Lecocq](https://discuss.elastic.co/u/Dan_Lecocq)\ **Post date:** [December 4, 2012, 8:42pm UTC](https://discuss.elastic.co/t/inefficient-and-unnecassary-rebalancing-when-re-opening-a-closed-index/9937/2 "2012-12-04T20:42:58Z")
Does each node still have one shard after the fact, they're just jumping between nodes? On Tuesday, December 4, 2012 11:03:38 AM UTC-8, Ivan Brusic wrote: > When re-opening a closed index, the cluster will move around the shards > even though the allocation should be even. > > Most recent example: re-opening a 4-shard/1-replica index on an 8 node > cluster. Each node had before closing the index exactly 1 shard. After > re-opening, 4 of the shards remained where they were, but the other 4 moved > to recovering on different nodes. The primary delegation for some shards > also changed after re-opening, so it is not just that the replicas are > recovering. > > Moving around just 1 replica of an index is a lot of network traffic. Is > there any way to avoid this situation? > > Cheers, > > Ivan -- ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [December 4, 2012, 9:49pm UTC](https://discuss.elastic.co/t/inefficient-and-unnecassary-rebalancing-when-re-opening-a-closed-index/9937/3 "2012-12-04T21:49:08Z")
Exactly. Each node in the cluster has the same amount of shards between all indices before the one index was closed, and after it was re-opened. Updates happen in bulk, so the index that was closed should not have any updates before it was closed (in the minutes, if not more). Hopefully that means the transaction log/gateway is clean. -- Ivan On Tue, Dec 4, 2012 at 12:42 PM, Dan Lecocq [dan@seomoz.org](mailto:dan@seomoz.org) wrote: > Does each node still have one shard after the fact, they're just jumping > between nodes? > > On Tuesday, December 4, 2012 11:03:38 AM UTC-8, Ivan Brusic wrote: > > > When re-opening a closed index, the cluster will move around the shards > > even though the allocation should be even. > > > > Most recent example: re-opening a 4-shard/1-replica index on an 8 node > > cluster. Each node had before closing the index exactly 1 shard. After > > re-opening, 4 of the shards remained where they were, but the other 4 moved > > to recovering on different nodes. The primary delegation for some shards > > also changed after re-opening, so it is not just that the replicas are > > recovering. > > > > Moving around just 1 replica of an index is a lot of network traffic. Is > > there any way to avoid this situation? > > > > Cheers, > > > > Ivan > > -- -- ---
**Author:** ![Dan\_Lecocq](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_lecocq/32/2565_2.png) [@Dan\_Lecocq](https://discuss.elastic.co/u/Dan_Lecocq)\ **Post date:** [December 4, 2012, 10:00pm UTC](https://discuss.elastic.co/t/inefficient-and-unnecassary-rebalancing-when-re-opening-a-closed-index/9937/4 "2012-12-04T22:00:03Z")
Are you using any of the `cluster.routing.allocation.awareness` settings? Though, I'd expect that to only affect how they're initially assigned :-/ On Tuesday, December 4, 2012 1:49:08 PM UTC-8, Ivan Brusic wrote: > Exactly. Each node in the cluster has the same amount of shards between > all indices before the one index was closed, and after it was re-opened. > > Updates happen in bulk, so the index that was closed should not have any > updates before it was closed (in the minutes, if not more). Hopefully that > means the transaction log/gateway is clean. > > -- > Ivan > > On Tue, Dec 4, 2012 at 12:42 PM, Dan Lecocq \<[d...@seomoz.org](mailto:d...@seomoz.org) \ > > > wrote: > > > Does each node still have one shard after the fact, they're just jumping > > between nodes? > > > > On Tuesday, December 4, 2012 11:03:38 AM UTC-8, Ivan Brusic wrote: > > > > > When re-opening a closed index, the cluster will move around the shards > > > even though the allocation should be even. > > > > > > Most recent example: re-opening a 4-shard/1-replica index on an 8 node > > > cluster. Each node had before closing the index exactly 1 shard. After > > > re-opening, 4 of the shards remained where they were, but the other 4 moved > > > to recovering on different nodes. The primary delegation for some shards > > > also changed after re-opening, so it is not just that the replicas are > > > recovering. > > > > > > Moving around just 1 replica of an index is a lot of network traffic. Is > > > there any way to avoid this situation? > > > > > > Cheers, > > > > > > Ivan > > > > -- -- ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [December 5, 2012, 9:50pm UTC](https://discuss.elastic.co/t/inefficient-and-unnecassary-rebalancing-when-re-opening-a-closed-index/9937/5 "2012-12-05T21:50:32Z")
Did not change the awareness settings. I did notice that gateway.expected\_nodes is set to 6 (on a 8 node cluster). Not the culprit, but something I should fix. Is this issue with the allocator or recovery? Does closing an index automatically flush the index? On Tue, Dec 4, 2012 at 2:00 PM, Dan Lecocq [dan@seomoz.org](mailto:dan@seomoz.org) wrote: > Are you using any of the `cluster.routing.allocation.awareness` settings? > Though, I'd expect that to only affect how they're initially assigned :-/ > > On Tuesday, December 4, 2012 1:49:08 PM UTC-8, Ivan Brusic wrote: > > > Exactly. Each node in the cluster has the same amount of shards between > > all indices before the one index was closed, and after it was re-opened. > > > > Updates happen in bulk, so the index that was closed should not have any > > updates before it was closed (in the minutes, if not more). Hopefully that > > means the transaction log/gateway is clean. > > > > -- > > Ivan > > > > On Tue, Dec 4, 2012 at 12:42 PM, Dan Lecocq [d...@seomoz.org](mailto:d...@seomoz.org) wrote: > > > > > Does each node still have one shard after the fact, they're just jumping > > > between nodes? > > > > > > On Tuesday, December 4, 2012 11:03:38 AM UTC-8, Ivan Brusic wrote: > > > > > > > When re-opening a closed index, the cluster will move around the shards > > > > even though the allocation should be even. > > > > > > > > Most recent example: re-opening a 4-shard/1-replica index on an 8 node > > > > cluster. Each node had before closing the index exactly 1 shard. After > > > > re-opening, 4 of the shards remained where they were, but the other 4 moved > > > > to recovering on different nodes. The primary delegation for some shards > > > > also changed after re-opening, so it is not just that the replicas are > > > > recovering. > > > > > > > > Moving around just 1 replica of an index is a lot of network traffic. > > > > Is there any way to avoid this situation? > > > > > > > > Cheers, > > > > > > > > Ivan > > > > > > -- > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:01am UTC](https://discuss.elastic.co/t/inefficient-and-unnecassary-rebalancing-when-re-opening-a-closed-index/9937/6 "2017-07-06T03:01:08Z")