When the number of documents exceeds 2 billion, the index status becomes RED

Elastic Stack Elasticsearch
1

Version: 7.8.1
Cause of failure: The number of important index documents in the 7.8.1 open source version of ES has reached the limit of 2147483519 in Lucene. The index status is red, and read and write operations cannot be performed.
Remark:

  1. This important index file has only one primary shard and one copy shard, neither of which can be read and written, and the index size is more than 200 G.
  2. Due to the rush to restore the business, directly back up the data files in the indices directory.

Hope idnex can return to normal state.

2

The best practices recommends no more than 20/50gb per shard.
I don't know if you can recover or if you have to reindex here to a better design. If you do, please follow the guidelines.

3

Thank you very much for your answer. The accident was caused by the company's R&D staff not having a good grasp of es. Is there really no way for me to recover the original index data now?

4

This version passed EOL a very long time ago and is not supported any more. This bug was fixed by the following PR almost 3 years ago:

2 Likes
5

I'd recommend recovering from a recent snapshot. I don't know of another way.

1 Like
6

This is unfortunate news for me because the company had no previous backups of that index.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.