# Routing: Equal distribution on all shards

**URL:** <https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531>\
**Category:** Elasticsearch\
**Created:** [May 30, 2019, 1:01pm UTC](https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531 "2019-05-30T13:01:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![pokaleshrey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pokaleshrey/32/48680_2.png) [@pokaleshrey](https://discuss.elastic.co/u/pokaleshrey)\
**Post date:** [May 30, 2019, 1:01pm UTC](https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531/1 "2019-05-30T13:01:42Z")

</div>

Hi, I have 5 unique routing keys "a","b","c","d","e" in my application.  
My ES setup consists of 5 shards.

When i check in current state (ver 6.6.2). My shard allocation is like this:

```
a	4	
b	2	
c	0	
d	3	
e	4

```

Note that shard number 1 is unallocated. Shard number 4 is overallocated.

**Query**  
Is there a way that i can ensure all my shards are distributed across 5 shards with one key each ?  
I understand there is a murmur hash algorithm which is involved in allocation.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 30, 2019, 9:47pm UTC](https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531/2 "2019-05-30T21:47:09Z")

</div>

The only way to force one allocation key per shard is to specify the routing when indexing things in Elasticsearch.

The default routing is based on a hash of the document, rather than a specific key.

---

<div class="post-metadata">

**Author:** ![pokaleshrey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pokaleshrey/32/48680_2.png) [@pokaleshrey](https://discuss.elastic.co/u/pokaleshrey)\
**Post date:** [May 31, 2019, 4:15am UTC](https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531/3 "2019-05-31T04:15:14Z")

</div>

Thanks for your reply.

I am not using the default routing, while indexing i do specify eg: routing=a.

My problem is when i finish indexing for all routing keys i have. Shard 1 remains unallocated.  
I need a way such that a,b,c,d,e, (having 5 shards in ES), documents for each routing key goes on separate shard.

I understand that it totally depends on the hash generate for the given keys.

**My base problem:**  
In above example say routing=a and routing=e goes on same shard i.e. 4.  
Assume a and e are types of documents, which have few fields in common. When i query documents with routing=a and on common fields, ES also checks all documents in routing=e lowering the performance extensively.  
[Check this thread](https://discuss.elastic.co/t/structured-hierarchy-repeating-fields-causing-low-performance/182482)

_Some issue which i see with shard allocation by routing_  
If i have 50shards and have 100 routing keys, it may happen that due to the nature of hash generated, some shards will remain unallocated for ever.

_Suggestion_:  
There could be a routing key to shard number mapping in the settings section which will give this control to the user. Such that user will decide which shard the routing key maps to.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 31, 2019, 4:59am UTC](https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531/4 "2019-05-31T04:59:48Z")

</div>

At the moment the routing key is hashed instead of the id. If you have large number of routing keys probability will give a reasonably even distribution. If you need more control and have few routing keys instead consider using multiple separate indices instead.

---

<div class="post-metadata">

**Author:** ![pokaleshrey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pokaleshrey/32/48680_2.png) [@pokaleshrey](https://discuss.elastic.co/u/pokaleshrey)\
**Post date:** [May 31, 2019, 5:19am UTC](https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531/5 "2019-05-31T05:19:21Z")

</div>

I already have 200 indexes, and the choosing of index based upon some factors is already manually handled in my code.  
Each index, inside it is having unique types, which i call the routing keys, which i wanted to distribute equally across shards (my original question).

This is like Types (routing keys) inside Types (my indexes).

I wonder why the control of choosing shard when routing is given, is not handed to user. I think this would add more flexibility in ES and better control to users.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 28, 2019, 5:19am UTC](https://discuss.elastic.co/t/routing-equal-distribution-on-all-shards/183531/6 "2019-06-28T05:19:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
