# Debugging Shard Allocation

**URL:** https://discuss.elastic.co/t/debugging-shard-allocation/2171
**Category:** Elasticsearch
**Created:** [June 9, 2015, 1:41am UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171 "2015-06-09T01:41:21Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![willhughes](https://avatars.discourse-cdn.com/v4/letter/w/9fc348/32.png) [@willhughes](https://discuss.elastic.co/u/willhughes)
#### Post date: [June 9, 2015, 1:41am UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171/1 "2015-06-09T01:41:21Z")

</div>

Is there any way to ask Elasticsearch why it allocated a shard to a specific node?

I've got a couple of different types of indexes in ES (some with small numbers of documents, others with millions more), and after rebooting various nodes to apply updates and, I've got an imbalanced cluster (all the million+ shards are allocated to one node).  
Nothing obvious in the cluster or index settings is standing out as to why it's not allocating to the others. If ES can tell me "Index A, Shard B cannot be allocated to Node X because of [y,z]" then that'd be great.

---

<div class="post-metadata">

### Author: ![Jason\_Wee](https://avatars.discourse-cdn.com/v4/letter/j/7ea924/32.png) [@Jason\_Wee](https://discuss.elastic.co/u/Jason_Wee)
#### Post date: [June 9, 2015, 10:02am UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171/2 "2015-06-09T10:02:41Z")

</div>

then you will need to read into the code, first the interface [https://github.com/elastic/elasticsearch/blob/v1.5.2/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/ShardsAllocator.java](https://github.com/elastic/elasticsearch/blob/v1.5.2/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/ShardsAllocator.java)

and then the concrete class, e.g. BalancedShardsAllocator.java [https://github.com/elastic/elasticsearch/blob/v1.5.2/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/BalancedShardsAllocator.java](https://github.com/elastic/elasticsearch/blob/v1.5.2/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/BalancedShardsAllocator.java)

and based on what concrete class, you see the logic maybe here [https://github.com/elastic/elasticsearch/blob/v1.5.2/src/main/java/org/elasticsearch/cluster/routing/allocation/AllocationService.java#L205](https://github.com/elastic/elasticsearch/blob/v1.5.2/src/main/java/org/elasticsearch/cluster/routing/allocation/AllocationService.java#L205)

hth

jason

---

<div class="post-metadata">

### Author: ![mosiddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mosiddi/32/577_2.png) [@mosiddi](https://discuss.elastic.co/u/mosiddi)
#### Post date: [June 9, 2015, 12:11pm UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171/3 "2015-06-09T12:11:13Z")

</div>

You can try doing [https://www.elastic.co/guide/en/elasticsearch/reference/master/cluster-reroute.html](https://www.elastic.co/guide/en/elasticsearch/reference/master/cluster-reroute.html) for one shard and use ?explain to see what ES tells u. This is obviously not solving the initial ask, but can help us get some data from ES if indeed it can't allocate to other nodes.

---

<div class="post-metadata">

### Author: ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)
#### Post date: [June 9, 2015, 12:47pm UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171/4 "2015-06-09T12:47:14Z")

</div>

> [@willhughes](#):
>
> I've got an imbalanced cluster (all the million+ shards are allocated to one node).

Elasticsearch's allocation weights all shards the same. See the pull request below for some of the reasoning as to why. At some point in the future it'll have more, better ways of tuning this stuff, but for now your best bet is setting `index.routing.allocation.total_shards_per_node` on the big indexes.

> <https://github.com/elastic/elasticsearch/pull/7171>
>
> From the docs:
> \`cluster.routing.allocation.balance.size\`::
> Defines a weight… factor for the number of similarly sized shards allocated
> on a node (float). Defaults to \`0.00f\` because it is new and we're not
> yet sure what a good default is. Raising this raises the tendency to
> equalize the number of similarly sized shards across all nodes in the
> cluster. "Similarly sized" means that this formula spits out the same
> number:
> size \<= 10^7^ bytes ? 0 : floor(log10(size))
> So 1GB and 5GB shards are "similar" and so are 1KB and 40MB.
> 
> Side effects:
> 1. Enables cluster info collection on all master eligible nodes so that
> they can make informed decisions based on shard sizes as soon as they get
> elected master. They don't have to wait for the job to return when they
> are elected.
> 2. Removes the disabled flag for the cluster info job. Its now complicate
> to know when you can disable it. Besides, the disk allocation decider is
> on by default now and that needs it too.
> 3. Reworks BalanceConfigurationTests quite a bit adding a test for the
> shard size balance and a test that random balance settings actually converge.
> 4. Fixes some of the cluster rebalance docs that were confusing on reread.
> 
> Closes #7155

---

<div class="post-metadata">

### Author: ![mosiddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mosiddi/32/577_2.png) [@mosiddi](https://discuss.elastic.co/u/mosiddi)
#### Post date: [June 9, 2015, 6:15pm UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171/5 "2015-06-09T18:15:25Z")

</div>

thanks @nik9000  
To ensure even allocation of primaries to all nodes, we are setting a bit higher-number to below setting -

> cluster.routing.allocation.balance.primary

This was done to overcome issue where one of our 5 data nodes had all primary shards of all indices. We used guess and trial number, do we see any side-effect of messing up with this number :)?

---

<div class="post-metadata">

### Author: ![willhughes](https://avatars.discourse-cdn.com/v4/letter/w/9fc348/32.png) [@willhughes](https://discuss.elastic.co/u/willhughes)
#### Post date: [June 10, 2015, 1:43am UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171/6 "2015-06-10T01:43:51Z")

</div>

Ah, nice. Thanks for that.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:08am UTC](https://discuss.elastic.co/t/debugging-shard-allocation/2171/7 "2017-07-06T00:08:44Z")

</div>


