# ArrayIndexOutOfBoundsException while indexing **URL:** \ **Category:** Elasticsearch\ **Created:** [February 20, 2018, 3:22am UTC](https://discuss.elastic.co/t/arrayindexoutofboundsexception-while-indexing/120550 "2018-02-20T03:22:33Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)\ **Post date:** [February 20, 2018, 3:22am UTC](https://discuss.elastic.co/t/arrayindexoutofboundsexception-while-indexing/120550/1 "2018-02-20T03:22:33Z")
Ran across this while indexing today. -65536 looks like it might be an overflow or something but not sure. This is the first attempt at indexing with large ngram settings (min=1, max=8) so that might be related. Any ideas as to where I should focus debugging? It happened 6-8 hours into our loading so it is not easily reproducible. This is ES 6.1.0, index has 165 shards, and a complex mapping. ```auto [2018-02-20T00:01:25,364][WARN][o.e.i.c.IndicesClusterStateService] [node] [[index][97]] marking and sending shard failed due to [shard failure, reason [already closed by tragic event on the index writer]] java.lang.ArrayIndexOutOfBoundsException: -65536 at org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:198) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:224) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.FreqProxTermsWriterPerField.writeProx(FreqProxTermsWriterPerField.java:80) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:184) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:786) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:280) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:436) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1530) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1506) ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 - ubuntu - 2017-10-13 16:12:42] at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:1002) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.index.engine.InternalEngine.indexIntoLucene(InternalEngine.java:946) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:815) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:732) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:701) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:667) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:548) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequest(TransportShardBulkAction.java:140) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:236) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.bulk.TransportShardBulkAction.performOnPrimary(TransportShardBulkAction.java:123) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:110) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:72) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:1033) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:1011) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:104) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:358) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:298) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:974) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:971) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) ~[elasticsearch-6.1.0.jar:6.1.0] at org.elasticsearch.index.shard.IndexShardOperationPermits$PermitAwareThreadedActionListener$1.doRun(IndexShardOperationPermits.java:305) ~[elasticsearch-6.1.0.jar:6.1.0] at ``` ---
**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\ **Post date:** [February 20, 2018, 4:55am UTC](https://discuss.elastic.co/t/arrayindexoutofboundsexception-while-indexing/120550/2 "2018-02-20T04:55:48Z")
With a minimum ngram of 1 and max of 8, I think you ran into [this situation](https://github.com/elastic/elasticsearch/issues/25887). As a result, ES introduced a new index-level setting called `index.max_ngram_diff` to prevent the min and max ngram settings to diverge too much. ---
**Author:** ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)\ **Post date:** [February 20, 2018, 3:27pm UTC](https://discuss.elastic.co/t/arrayindexoutofboundsexception-while-indexing/120550/3 "2018-02-20T15:27:26Z")
Thanks val, I knew of this and suspected it. I don't think 8 ngram max is that crazy of a number considering the issue that promoted these soft limits was 7k. Ultimately I would like to know what the actual issue is, too many terms is not a sufficient answer. Are we overflowing a position or offset counter? What ultimately leads to this exception being thrown? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 20, 2018, 3:27pm UTC](https://discuss.elastic.co/t/arrayindexoutofboundsexception-while-indexing/120550/4 "2018-03-20T15:27:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.