# Limitation of 2,147,483,647 terms per segment index in Lucene
**URL:** https://discuss.elastic.co/t/limitation-of-2-147-483-647-terms-per-segment-index-in-lucene/18401
**Category:** Elasticsearch
**Created:** [June 30, 2014, 3:49pm UTC](https://discuss.elastic.co/t/limitation-of-2-147-483-647-terms-per-segment-index-in-lucene/18401 "2014-06-30T15:49:32Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Peter\_Portante](https://discuss.elastic.co/u/Peter_Portante)
#### Post date: [June 30, 2014, 3:49pm UTC](https://discuss.elastic.co/t/limitation-of-2-147-483-647-terms-per-segment-index-in-lucene/18401/1 "2014-06-30T15:49:32Z")
Is there a way to recover a segment index of a shard that has exceeded
Lucene's 2^31 limit?
Thanks,
-peter
[2014-06-30 10:53:02,187][WARN][indices.cluster] [Patriots]
[vos][0] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[vos][0] failed recovery
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:185)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IllegalArgumentException: Too many documents,
composite IndexReaders cannot exceed 2147483647
at
org.apache.lucene.index.BaseCompositeReader.(BaseCompositeReader.java:77)
at
org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:369)
at
org.apache.lucene.index.StandardDirectoryReader.(StandardDirectoryReader.java:43)
at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:115)
at
org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:385)
at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:112)
at
org.apache.lucene.search.SearcherManager.(SearcherManager.java:89)
at
org.elasticsearch.index.engine.internal.InternalEngine.buildSearchManager(InternalEngine.java:1364)
at
org.elasticsearch.index.engine.internal.InternalEngine.start(InternalEngine.java:291)
at
org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryPrepareForTranslog(InternalIndexShard.java:709)
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:204)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132)
... 3 more
[2014-06-30 10:53:02,213][WARN][cluster.action.shard] [Patriots]
[vos][0] sending failed shard for [vos][0], node[bS9Lp\_a9QZOjiab23Ztk4A],
[P], s[INITIALIZING], indexUUID [a0\_HJrlgQq-UNCwL2QiVbg], reason [Failed to
start shard, message [IndexShardGatewayRecoveryException[[vos][0] failed
recovery]; nested: IllegalArgumentException[Too many documents, composite
IndexReaders cannot exceed 2147483647]; ]]
[2014-06-30 10:53:02,213][WARN][cluster.action.shard] [Patriots]
[vos][0] received shard failed for [vos][0], node[bS9Lp\_a9QZOjiab23Ztk4A],
[P], s[INITIALIZING], indexUUID [a0\_HJrlgQq-UNCwL2QiVbg], reason [Failed to
start shard, message [IndexShardGatewayRecoveryException[[vos][0] failed
recovery]; nested: IllegalArgumentException[Too many documents, composite
IndexReaders cannot exceed 2147483647]; ]]
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f5e2a408-f1e5-469a-9bcc-679f399dbe9c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f5e2a408-f1e5-469a-9bcc-679f399dbe9c%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@simonw\_3](https://discuss.elastic.co/u/simonw_3)
#### Post date: [July 2, 2014, 7:53am UTC](https://discuss.elastic.co/t/limitation-of-2-147-483-647-terms-per-segment-index-in-lucene/18401/2 "2014-07-02T07:53:52Z")
Peter, thanks so much for raising this. This looks aweful! I think we
should move this into an issue on [1] (please feel free to create one) IMO
we should aim to name the issue in a way to prevent this from happening
altogether. Along the lines we should help you to recover but I don't know
how tricky it will be. Lets start with the issue!!
simon
[1] [Issues · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues)
On Monday, June 30, 2014 5:49:32 PM UTC+2, Peter Portante wrote:
> Is there a way to recover a segment index of a shard that has exceeded
> Lucene's 2^31 limit?
>
> Thanks,
> -peter
>
> [2014-06-30 10:53:02,187][WARN][indices.cluster] [Patriots]
> [vos][0] failed to start shard
> org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> [vos][0] failed recovery
> at
> org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:185)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.IllegalArgumentException: Too many documents,
> composite IndexReaders cannot exceed 2147483647
> at
> org.apache.lucene.index.BaseCompositeReader.(BaseCompositeReader.java:77)
> at
> org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:369)
> at
> org.apache.lucene.index.StandardDirectoryReader.(StandardDirectoryReader.java:43)
> at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:115)
> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:385)
> at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:112)
> at
> org.apache.lucene.search.SearcherManager.(SearcherManager.java:89)
> at
> org.elasticsearch.index.engine.internal.InternalEngine.buildSearchManager(InternalEngine.java:1364)
> at
> org.elasticsearch.index.engine.internal.InternalEngine.start(InternalEngine.java:291)
> at
> org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryPrepareForTranslog(InternalIndexShard.java:709)
> at
> org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:204)
> at
> org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132)
> ... 3 more
> [2014-06-30 10:53:02,213][WARN][cluster.action.shard] [Patriots]
> [vos][0] sending failed shard for [vos][0], node[bS9Lp\_a9QZOjiab23Ztk4A],
> [P], s[INITIALIZING], indexUUID [a0\_HJrlgQq-UNCwL2QiVbg], reason [Failed to
> start shard, message [IndexShardGatewayRecoveryException[[vos][0] failed
> recovery]; nested: IllegalArgumentException[Too many documents, composite
> IndexReaders cannot exceed 2147483647]; ]]
> [2014-06-30 10:53:02,213][WARN][cluster.action.shard] [Patriots]
> [vos][0] received shard failed for [vos][0], node[bS9Lp\_a9QZOjiab23Ztk4A],
> [P], s[INITIALIZING], indexUUID [a0\_HJrlgQq-UNCwL2QiVbg], reason [Failed to
> start shard, message [IndexShardGatewayRecoveryException[[vos][0] failed
> recovery]; nested: IllegalArgumentException[Too many documents, composite
> IndexReaders cannot exceed 2147483647]; ]]
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/01253247-c16b-44cb-aa61-d02bc20f44eb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/01253247-c16b-44cb-aa61-d02bc20f44eb%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:18am UTC](https://discuss.elastic.co/t/limitation-of-2-147-483-647-terms-per-segment-index-in-lucene/18401/3 "2017-07-06T01:18:29Z")