# Max\_docs\_per\_index exceeded

**URL:** <https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994>\
**Category:** Elasticsearch\
**Created:** [January 12, 2023, 7:38am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994 "2023-01-12T07:38:41Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dimitri\_F](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dimitri\_F](https://discuss.elastic.co/u/Dimitri_F)\
**Post date:** [January 12, 2023, 7:38am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/1 "2023-01-12T07:38:41Z")

</div>

Hello. I use Elasticsearch with Graylog. During this week I noticed that my Elasticsearch server is running out of disk space. It’s configured to store 240 indices. Max\_docs\_per\_index is 20 million documents.

Having checked the size of indices on disk, I quickly noticed that one of them is significantly larger than all others. It contained over 900 mln documents when it finally became inactive. What do you think could be the cause?

It's not the first time it's happening. Last November, one of indices grew to 500 mln docs and then turned inactive and all the next indices didn't exceed the 20 million limit till today.

Graylog 4.2.13 and Elasticsearch 7.10.1 running on Debian.

Thank you.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/7/97864979ab2fb09c4b1b99f653b5c8de7ec19b91.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 12, 2023, 7:38am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/2 "2023-01-12T07:38:41Z")

</div>

Elasticsearch 7.10 is [EOL](https://www.elastic.co/support/eol) and no longer supported. Please upgrade ASAP.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 12, 2023, 7:48am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/3 "2023-01-12T07:48:11Z")

</div>

Welcome to our community! 😃

> [@Dimitri\_F](#):
>
> Max\_docs\_per\_index

That seems to be a graylog setting, not an Elasticsearch one, so I am not 100% we can help here sorry.

---

<div class="post-metadata">

**Author:** ![Dimitri\_F](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dimitri\_F](https://discuss.elastic.co/u/Dimitri_F)\
**Post date:** [January 12, 2023, 9:03am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/4 "2023-01-12T09:03:50Z")

</div>

Thank you, Mark. You gave me a direction. Here's what I found in Graylog log files:

`Caused by: org.graylog.shaded.elasticsearch7.org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=validation_exception, reason=Validation Failed: 1: this action would add [4] total shards, but this cluster currently has [997]/[1000] maximum shards open;]`

I've set `cluster.max_shards_per_node` to 3000. Is it a valid solution? Is there anything else I should do? It's a single-node ES instance.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 12, 2023, 9:07am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/5 "2023-01-12T09:07:33Z")

</div>

It's not ideal as it creates heap pressure.

---

<div class="post-metadata">

**Author:** ![Dimitri\_F](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dimitri\_F](https://discuss.elastic.co/u/Dimitri_F)\
**Post date:** [January 12, 2023, 9:10am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/6 "2023-01-12T09:10:57Z")

</div>

Should I decrease the number of indices, while raising their size limit, then?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 12, 2023, 9:14am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/7 "2023-01-12T09:14:35Z")

</div>

Based on the screenshot it looks like each index has 4 primary shards and generally are around 6GB in size. This is quite small (1.5GB) and will as you see result in a lot of shards. If Graylog limits indices by document count it probably would be reasonable to increase the limit by a factor of between 5 and 10 in order to get the average shard size up.

---

<div class="post-metadata">

**Author:** ![Dimitri\_F](https://avatars.discourse-cdn.com/v4/letter/d/ba9def/32.png) [@Dimitri\_F](https://discuss.elastic.co/u/Dimitri_F)\
**Post date:** [January 12, 2023, 9:25am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/8 "2023-01-12T09:25:07Z")

</div>

Thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 9, 2023, 9:25am UTC](https://discuss.elastic.co/t/max-docs-per-index-exceeded/322994/9 "2023-02-09T09:25:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
