# Balancing indexes by priority

**URL:** <https://discuss.elastic.co/t/balancing-indexes-by-priority/318110>\
**Category:** Elasticsearch\
**Created:** [November 3, 2022, 12:30pm UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110 "2022-11-03T12:30:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![driveirk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/driveirk/32/102253_2.png) [@driveirk](https://discuss.elastic.co/u/driveirk)\
**Post date:** [November 3, 2022, 12:30pm UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/1 "2022-11-03T12:30:45Z")

</div>

Very often, we get so that all new indices converge on one node and the node starts to reject.  
Now we use ILM and each index is given a priority.  
I want to configure so that indexes with equal priority are placed evenly across all servers.

For example now:  
This is the number of indexes of the same priority that are written to for each server.

```auto
    343 ess04
    353 ess07
    527 ess02
    531 ess03
    568 ess06

```

This is the number of indexes of the same priority that are written to for each node.

```auto
    165 ess02-1
    168 ess04-2
    170 ess07-3
    172 ess06-3
    174 ess02-3
    175 ess03-1
    175 ess04-3
    178 ess03-2
    178 ess03-3
    183 ess07-1
    188 ess02-2
    188 ess06-2
    208 ess06-1

```

distribution of all indices

```auto
415 ess06-3
415 ess03-2
415 ess03-3
415 ess06-1
415 ess07-3
416 ess07-1
416 ess06-2
415 ess04-3
415 ess03-1
416 ess02-3
416 ess04-2
415 ess02-2
415 ess02-1

```

As you can see, balancing does not work well in this regard. And the ess06 server is now under increased load. How can I fix the situation?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 3, 2022, 9:58pm UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/2 "2022-11-03T21:58:26Z")

</div>

If you are referring to `index.priority` this relates to recovery only, as per [the docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/recovery-prioritization.html). Elasticsearch balances indices based on the shard count on each node, followed by how much disk space the nodes have.

Are you using tiering with ILM?

---

<div class="post-metadata">

**Author:** ![driveirk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/driveirk/32/102253_2.png) [@driveirk](https://discuss.elastic.co/u/driveirk)\
**Post date:** [November 3, 2022, 11:31pm UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/3 "2022-11-03T23:31:11Z")

</div>

I just don't know how else to find the indexes that are currently being logged when using `data-stream`, I have the highest `index.priority` in the `hot` phase, after the rollover the `warm` phase immediately follows.  
All nodes have the same roles. The task is to distribute all the indexes in which the record is made evenly throughout the cluster.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 4, 2022, 12:43am UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/4 "2022-11-04T00:43:27Z")

</div>

What's the solution you are trying to reach here?

---

<div class="post-metadata">

**Author:** ![driveirk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/driveirk/32/102253_2.png) [@driveirk](https://discuss.elastic.co/u/driveirk)\
**Post date:** [November 4, 2022, 8:10am UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/5 "2022-11-04T08:10:41Z")

</div>

I want to know how people do balancing, I thought the developers could tell me how to solve this problem.  
Maybe someone has already encountered this problem and solved it.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 4, 2022, 8:13am UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/6 "2022-11-04T08:13:13Z")

</div>

Your last lot of numbers show that the node you are concerned about has pretty much the same number of shards as all the other nodes, that is how Elasticsearch balances things.

Are you seeing all shards for a new index being allocated on a single node?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 4, 2022, 8:13am UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/7 "2022-11-04T08:13:47Z")

</div>

How many indices are you actively indexing into? How many indices do you have in total? How many primary shards do these have?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 2, 2022, 8:14am UTC](https://discuss.elastic.co/t/balancing-indexes-by-priority/318110/8 "2022-12-02T08:14:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
