# Elasticsearch throwing number of documents in the index cannot exceed 2147483519

**URL:** <https://discuss.elastic.co/t/elasticsearch-throwing-number-of-documents-in-the-index-cannot-exceed-2147483519/242512>\
**Category:** Elasticsearch\
**Created:** [July 24, 2020, 12:44pm UTC](https://discuss.elastic.co/t/elasticsearch-throwing-number-of-documents-in-the-index-cannot-exceed-2147483519/242512 "2020-07-24T12:44:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ganesh2303](https://avatars.discourse-cdn.com/v4/letter/g/57b2e6/32.png) [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\
**Post date:** [July 24, 2020, 12:44pm UTC](https://discuss.elastic.co/t/elasticsearch-throwing-number-of-documents-in-the-index-cannot-exceed-2147483519/242512/1 "2020-07-24T12:44:37Z")

</div>

HI Team,  
I'm using monthly index to store all my data in elasticsearch. i have enabled DLQ to capture the unformatted log information and in that im getting capture below error.

> elasticsearch ±Could not index event to Elasticsearch. status: 400, action: ["index", {:\_id=\>nil, :\_index=\>"xx-logstash-2020.07", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x1f452fb3], response: {"index"=\>{"\_index"=\>"xx-logstash-2020.07", "\_type"=\>"doc", "\_id"=\>"\_XRRP3MBt9usZDg0br2M", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"number of documents in the index cannot exceed 2147483519"}}}c éÿÿÿÿNc 2020-07-11T19:19:28.809Z

is that any limit available to index data in single index?

---

<div class="post-metadata">

**Author:** ![Bernt\_Rostad](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@Bernt\_Rostad](https://discuss.elastic.co/u/Bernt_Rostad)\
**Post date:** [July 24, 2020, 2:12pm UTC](https://discuss.elastic.co/t/elasticsearch-throwing-number-of-documents-in-the-index-cannot-exceed-2147483519/242512/2 "2020-07-24T14:12:49Z")

</div>

This is a limitation of the underlying Lucene engine which only accepts 2^31 documents in an index. So you're basically stuck, that index won't be able to store more documents.

But since **one Lucene index = one Elasticsearch shard** you could avoid this problem in the future by adding more primary shards to your Elasticsearch logstash indices, since each independent shard can store 2 billion documents.

Personally, I would probably change to weekly indices instead since 2+ billion documents is a lot to search through if, say, you just want to aggregate or search on the last few days or weeks of log data. I rarely do a full month of aggregations, and if I need that I could still do it by aggregating over 4 smaller weekly indices rather than 1 big monthly.

The choice is yours, either go for weekly indices or add more primary shards to the monthly index so that it can store more documents. Good luck!

---

<div class="post-metadata">

**Author:** ![Ganesh2303](https://avatars.discourse-cdn.com/v4/letter/g/57b2e6/32.png) [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\
**Post date:** [July 24, 2020, 2:17pm UTC](https://discuss.elastic.co/t/elasticsearch-throwing-number-of-documents-in-the-index-cannot-exceed-2147483519/242512/3 "2020-07-24T14:17:27Z")

</div>

@Bernt_Rostad Thanks for your input and we consider your point and implement those in my configuration.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 21, 2020, 2:17pm UTC](https://discuss.elastic.co/t/elasticsearch-throwing-number-of-documents-in-the-index-cannot-exceed-2147483519/242512/4 "2020-08-21T14:17:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
