Error for too many documents, API says we are still below the limit

Elastic Stack Elasticsearch
1

While indexing new documents we've recently started getting errors for having too many (java.lang.IllegalArgumentException: number of documents in the index cannot exceed 2147483519).

We are aware of this limit, but looking at our numbers per shard, we aren't quite sure how we are hitting it? We are still quite a few docs away.

/_cat/shards?pretty results in (edited for readability):
our-index 2 r STARTED 1898146592
our-index 2 p STARTED 1898146572
our-index 2 r STARTED 1898146592
our-index 3 r STARTED 1402285175
our-index 3 p STARTED 1402285818
our-index 3 r STARTED 1402284441
our-index 4 p STARTED 677430317
our-index 4 r STARTED 677430212
our-index 4 r STARTED 677429764
our-index 1 p STARTED 688333390
our-index 1 r STARTED 688333305
our-index 1 r STARTED 685486884
our-index 0 r STARTED 1371042072
our-index 0 r STARTED 1371041952
our-index 0 p STARTED 1371042386

We've been thinking this might be due to nested documents, but if that's the case we have yet to find an API that lists a total number of docs per shard.

Any ideas would be greatly appreciated?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.