# Shard rebalancing

**URL:** https://discuss.elastic.co/t/shard-rebalancing/19057
**Category:** Elasticsearch
**Created:** [August 3, 2014, 6:45pm UTC](https://discuss.elastic.co/t/shard-rebalancing/19057 "2014-08-03T18:45:44Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![ElasticSearch\_Users\_](https://avatars.discourse-cdn.com/v4/letter/e/b4bc9f/32.png) [@ElasticSearch\_Users\_](https://discuss.elastic.co/u/ElasticSearch_Users_)
#### Post date: [August 3, 2014, 6:45pm UTC](https://discuss.elastic.co/t/shard-rebalancing/19057/1 "2014-08-03T18:45:44Z")

</div>

What is the behavior of ES when it comes to shard sizes? Does it do  
automatic shard rebalancing at any point of time? If so, is it also  
controlled through an API?

How can I know if the shards are changing in the background? If I do not  
add any new node or change any cluster configuration once indexing has  
started, is there any pattern to this behavior? Please let me know.

Thanks,  
Sandeep

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9f3dee22-0c33-4446-a0dc-eaf1a314d2c4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9f3dee22-0c33-4446-a0dc-eaf1a314d2c4%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [August 3, 2014, 11:43pm UTC](https://discuss.elastic.co/t/shard-rebalancing/19057/2 "2014-08-03T23:43:34Z")

</div>

Shard size will depend entirely on how many shards you've set and how big  
the index is.  
Allocation of data to shards happens in a round-robin manner, so balancing  
isn't needed.

What do you mean by shards changing in the background?

Regards,  
Mark Walkom

Infrastructure Engineer  
Campaign Monitor  
email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)  
web: [www.campaignmonitor.com](http://www.campaignmonitor.com)

On 4 August 2014 04:45, 'Sandeep Ramesh Khanzode' via elasticsearch \<  
[elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)\> wrote:

> What is the behavior of ES when it comes to shard sizes? Does it do  
> automatic shard rebalancing at any point of time? If so, is it also  
> controlled through an API?
> 
> How can I know if the shards are changing in the background? If I do not  
> add any new node or change any cluster configuration once indexing has  
> started, is there any pattern to this behavior? Please let me know.
> 
> Thanks,  
> Sandeep
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/9f3dee22-0c33-4446-a0dc-eaf1a314d2c4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9f3dee22-0c33-4446-a0dc-eaf1a314d2c4%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/9f3dee22-0c33-4446-a0dc-eaf1a314d2c4%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/9f3dee22-0c33-4446-a0dc-eaf1a314d2c4%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624aCRon%2B1%2BFWKnC7tkKU5oMYyHiONGaOiTJs%3DCHhE%3DezoA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624aCRon%2B1%2BFWKnC7tkKU5oMYyHiONGaOiTJs%3DCHhE%3DezoA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)
#### Post date: [August 4, 2014, 1:08pm UTC](https://discuss.elastic.co/t/shard-rebalancing/19057/3 "2014-08-04T13:08:42Z")

</div>

On Sun, Aug 3, 2014 at 7:43 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)  
wrote:

> Shard size will depend entirely on how many shards you've set and how big  
> the index is.  
> Allocation of data to shards happens in a round-robin manner, so balancing  
> isn't needed.
> 
> What do you mean by shards changing in the background?

The shards aren't assigned round robin - they are assigned to minimize a  
cluster balance function. The cluster balance function is a combination of  
three independent functions described here:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

None of them take into account shard size. I'm working on one right now  
that does.

For now your best bet is to set  
cluster.routing.allocation.balance.index pretty high, like 80% of the sum.  
That'll make all shards of a given index are smoothed across all nodes.

The new one will try to balance the count of shards of similar size across  
all the nodes. That keeps one node from filling up while the others only  
have small shards, for example. It should also spread load more evenly  
when the indexes are different sizes.

Nik

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3ah7gtmJJkHKmi9hNncU%3D8E5%3DcdLrb2LjyrBaeOZnyiw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3ah7gtmJJkHKmi9hNncU%3D8E5%3DcdLrb2LjyrBaeOZnyiw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:11am UTC](https://discuss.elastic.co/t/shard-rebalancing/19057/4 "2017-07-06T01:11:03Z")

</div>


