# Merge Error with lucene 4.1

**URL:** <https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873>\
**Category:** Elasticsearch\
**Created:** [February 25, 2013, 6:54am UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873 "2013-02-25T06:54:46Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![taras](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/taras/32/507_2.png) [@taras](https://discuss.elastic.co/u/taras)\
**Post date:** [February 25, 2013, 6:54am UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/1 "2013-02-25T06:54:46Z")

</div>

While testing the master build with lucene 4.1, I got a permanent merge  
error filling up my logs.  
Has anyone else seen it? Anything I can provide to help debug?

[2013-02-24 22:43:38,863][WARN][index.merge.scheduler] [Enchantress]  
[events][0] failed to merge  
java.lang.ArrayIndexOutOfBoundsException  
at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
at  
org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
at  
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
at  
org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
at  
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
at  
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
at  
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)

Thanks!

-Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\
**Post date:** [February 25, 2013, 8:42am UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/2 "2013-02-25T08:42:20Z")

</div>

hey,

can you give some more background here, like did you start this on an old  
index ie. an index you created with a previous version of ES / Lucene? Can  
you reproduce this?

simon

On Monday, February 25, 2013 7:54:46 AM UTC+1, Taras Shkvarchuk wrote:

> While testing the master build with lucene 4.1, I got a permanent merge  
> error filling up my logs.  
> Has anyone else seen it? Anything I can provide to help debug?
> 
> [2013-02-24 22:43:38,863][WARN][index.merge.scheduler] [Enchantress]  
> [events][0] failed to merge  
> java.lang.ArrayIndexOutOfBoundsException  
> at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
> at  
> org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
> at  
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
> at  
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
> at  
> org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
> at  
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
> at  
> org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
> at  
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
> 
> Thanks!
> 
> -Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\
**Post date:** [February 25, 2013, 2:22pm UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/3 "2013-02-25T14:22:31Z")

</div>

Hey Taras,

it would be awesome if you could provide some more infos about you problems  
so we can fix it quickly

simon

On Monday, February 25, 2013 9:42:20 AM UTC+1, simonw wrote:

> hey,
> 
> can you give some more background here, like did you start this on an old  
> index ie. an index you created with a previous version of ES / Lucene? Can  
> you reproduce this?
> 
> simon
> 
> On Monday, February 25, 2013 7:54:46 AM UTC+1, Taras Shkvarchuk wrote:
> 
> > While testing the master build with lucene 4.1, I got a permanent merge  
> > error filling up my logs.  
> > Has anyone else seen it? Anything I can provide to help debug?
> > 
> > [2013-02-24 22:43:38,863][WARN][index.merge.scheduler] [Enchantress]  
> > [events][0] failed to merge  
> > java.lang.ArrayIndexOutOfBoundsException  
> > at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
> > at  
> > org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
> > at  
> > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
> > at  
> > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
> > at  
> > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
> > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
> > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
> > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
> > at  
> > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
> > at  
> > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
> > at  
> > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
> > 
> > Thanks!
> > 
> > -Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![taras](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/taras/32/507_2.png) [@taras](https://discuss.elastic.co/u/taras)\
**Post date:** [February 25, 2013, 6:10pm UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/4 "2013-02-25T18:10:18Z")

</div>

Not sure what type of info you're looking for, so you can e-mail me for  
more details, I can probably even provide the index in question for  
debugging.  
There are 420K root level documents as all of which have small nested  
documents. Re-indexing nearly the same data resulted in an index without  
corruption.

Here is the \_status and \_segments call.  
{  
"ok" : true,  
"\_shards" : {  
"total" : 1,  
"successful" : 1,  
"failed" : 0  
},  
"indices" : {  
"events" : {  
"index" : {  
"primary\_size" : "673.3mb",  
"primary\_size\_in\_bytes" : 706043580,  
"size" : "673.3mb",  
"size\_in\_bytes" : 706043580  
},  
"translog" : {  
"operations" : 0  
},  
"docs" : {  
"num\_docs" : 1869131,  
"max\_doc" : 2263281,  
"deleted\_docs" : 394150  
},  
"merges" : {  
"current" : 0,  
"current\_docs" : 0,  
"current\_size" : "0b",  
"current\_size\_in\_bytes" : 0,  
"total" : 1,  
"total\_time" : "3.7s",  
"total\_time\_in\_millis" : 3776,  
"total\_docs" : 2263281,  
"total\_size" : "0b",  
"total\_size\_in\_bytes" : 0  
},  
"refresh" : {  
"total" : 1,  
"total\_time" : "0s",  
"total\_time\_in\_millis" : 0  
},  
"flush" : {  
"total" : 1,  
"total\_time" : "18ms",  
"total\_time\_in\_millis" : 18  
},  
"shards" : {  
"0" : [ {  
"routing" : {  
"state" : "STARTED",  
"primary" : true,  
"node" : "vwbSeFjtQ3SqdbJzh8V-uA",  
"relocating\_node" : null,  
"shard" : 0,  
"index" : "events"  
},  
"state" : "STARTED",  
"index" : {  
"size" : "673.3mb",  
"size\_in\_bytes" : 706043580  
},  
"translog" : {  
"id" : 1361432592857,  
"operations" : 0  
},  
"docs" : {  
"num\_docs" : 1869131,  
"max\_doc" : 2263281,  
"deleted\_docs" : 394150  
},  
"merges" : {  
"current" : 0,  
"current\_docs" : 0,  
"current\_size" : "0b",  
"current\_size\_in\_bytes" : 0,  
"total" : 1,  
"total\_time" : "3.7s",  
"total\_time\_in\_millis" : 3776,  
"total\_docs" : 2263281,  
"total\_size" : "0b",  
"total\_size\_in\_bytes" : 0  
},  
"refresh" : {  
"total" : 1,  
"total\_time" : "0s",  
"total\_time\_in\_millis" : 0  
},  
"flush" : {  
"total" : 1,  
"total\_time" : "18ms",  
"total\_time\_in\_millis" : 18  
}  
} ]  
}  
}  
}

\_segments

{  
"ok" : true,  
"\_shards" : {  
"total" : 1,  
"successful" : 1,  
"failed" : 0  
},  
"indices" : {  
"events" : {  
"shards" : {  
"0" : [ {  
"routing" : {  
"state" : "STARTED",  
"primary" : true,  
"node" : "vwbSeFjtQ3SqdbJzh8V-uA"  
},  
"num\_committed\_segments" : 20,  
"num\_search\_segments" : 20,  
"segments" : {  
"\_4qm" : {  
"generation" : 6142,  
"num\_docs" : 6730,  
"deleted\_docs" : 67313,  
"size" : "29.8mb",  
"size\_in\_bytes" : 31342776,  
"committed" : true,  
"search" : true  
},  
"\_ukk" : {  
"generation" : 39620,  
"num\_docs" : 579895,  
"deleted\_docs" : 232003,  
"size" : "203.9mb",  
"size\_in\_bytes" : 213859775,  
"committed" : true,  
"search" : true  
},  
"\_1o32" : {  
"generation" : 77870,  
"num\_docs" : 1131181,  
"deleted\_docs" : 71964,  
"size" : "389.9mb",  
"size\_in\_bytes" : 408880404,  
"committed" : true,  
"search" : true  
},  
"\_1o3e" : {  
"generation" : 77882,  
"num\_docs" : 17179,  
"deleted\_docs" : 15495,  
"size" : "9.9mb",  
"size\_in\_bytes" : 10401244,  
"committed" : true,  
"search" : true  
},  
"\_1qse" : {  
"generation" : 81374,  
"num\_docs" : 64627,  
"deleted\_docs" : 1870,  
"size" : "16.5mb",  
"size\_in\_bytes" : 17305817,  
"committed" : true,  
"search" : true  
},  
"\_1qyf" : {  
"generation" : 81591,  
"num\_docs" : 9475,  
"deleted\_docs" : 2073,  
"size" : "3.7mb",  
"size\_in\_bytes" : 3894229,  
"committed" : true,  
"search" : true  
},  
"\_1r7a" : {  
"generation" : 81910,  
"num\_docs" : 15972,  
"deleted\_docs" : 2375,  
"size" : "5mb",  
"size\_in\_bytes" : 5267040,  
"committed" : true,  
"search" : true  
},  
"\_1rck" : {  
"generation" : 82100,  
"num\_docs" : 9818,  
"deleted\_docs" : 6,  
"size" : "3.5mb",  
"size\_in\_bytes" : 3753253,  
"committed" : true,  
"search" : true  
},  
"\_1re0" : {  
"generation" : 82152,  
"num\_docs" : 16651,  
"deleted\_docs" : 787,  
"size" : "4.6mb",  
"size\_in\_bytes" : 4896408,  
"committed" : true,  
"search" : true  
},  
"\_1rhz" : {  
"generation" : 82295,  
"num\_docs" : 14006,  
"deleted\_docs" : 264,  
"size" : "4.7mb",  
"size\_in\_bytes" : 5016838,  
"committed" : true,  
"search" : true  
},  
"\_1ri0" : {  
"generation" : 82296,  
"num\_docs" : 479,  
"deleted\_docs" : 0,  
"size" : "162.7kb",  
"size\_in\_bytes" : 166651,  
"committed" : true,  
"search" : true  
},  
"\_1ri4" : {  
"generation" : 82300,  
"num\_docs" : 266,  
"deleted\_docs" : 0,  
"size" : "110.1kb",  
"size\_in\_bytes" : 112763,  
"committed" : true,  
"search" : true  
},  
"\_1ri6" : {  
"generation" : 82302,  
"num\_docs" : 250,  
"deleted\_docs" : 0,  
"size" : "109.3kb",  
"size\_in\_bytes" : 111965,  
"committed" : true,  
"search" : true  
},  
"\_1ri8" : {  
"generation" : 82304,  
"num\_docs" : 153,  
"deleted\_docs" : 0,  
"size" : "85.4kb",  
"size\_in\_bytes" : 87460,  
"committed" : true,  
"search" : true  
},  
"\_1ria" : {  
"generation" : 82306,  
"num\_docs" : 549,  
"deleted\_docs" : 0,  
"size" : "190.7kb",  
"size\_in\_bytes" : 195340,  
"committed" : true,  
"search" : true  
},  
"\_1riu" : {  
"generation" : 82326,  
"num\_docs" : 261,  
"deleted\_docs" : 0,  
"size" : "106kb",  
"size\_in\_bytes" : 108604,  
"committed" : true,  
"search" : true  
},  
"\_1rj6" : {  
"generation" : 82338,  
"num\_docs" : 533,  
"deleted\_docs" : 0,  
"size" : "191.1kb",  
"size\_in\_bytes" : 195748,  
"committed" : true,  
"search" : true  
},  
"\_1rj8" : {  
"generation" : 82340,  
"num\_docs" : 276,  
"deleted\_docs" : 0,  
"size" : "119.4kb",  
"size\_in\_bytes" : 122299,  
"committed" : true,  
"search" : true  
},  
"\_1rjb" : {  
"generation" : 82343,  
"num\_docs" : 251,  
"deleted\_docs" : 0,  
"size" : "100.7kb",  
"size\_in\_bytes" : 103129,  
"committed" : true,  
"search" : true  
},  
"\_1rjs" : {  
"generation" : 82360,  
"num\_docs" : 579,  
"deleted\_docs" : 0,  
"size" : "207.2kb",  
"size\_in\_bytes" : 212180,  
"committed" : true,  
"search" : true  
}  
}  
} ]  
}  
}  
}  
}

On Monday, February 25, 2013 6:22:31 AM UTC-8, simonw wrote:

> Hey Taras,
> 
> it would be awesome if you could provide some more infos about you  
> problems so we can fix it quickly
> 
> simon
> 
> On Monday, February 25, 2013 9:42:20 AM UTC+1, simonw wrote:
> 
> > hey,
> > 
> > can you give some more background here, like did you start this on an old  
> > index ie. an index you created with a previous version of ES / Lucene? Can  
> > you reproduce this?
> > 
> > simon
> > 
> > On Monday, February 25, 2013 7:54:46 AM UTC+1, Taras Shkvarchuk wrote:
> > 
> > > While testing the master build with lucene 4.1, I got a permanent merge  
> > > error filling up my logs.  
> > > Has anyone else seen it? Anything I can provide to help debug?
> > > 
> > > [2013-02-24 22:43:38,863][WARN][index.merge.scheduler]  
> > > [Enchantress] [events][0] failed to merge  
> > > java.lang.ArrayIndexOutOfBoundsException  
> > > at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
> > > at  
> > > org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
> > > at  
> > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
> > > at  
> > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
> > > at  
> > > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
> > > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
> > > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
> > > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
> > > at  
> > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
> > > at  
> > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
> > > at  
> > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
> > > 
> > > Thanks!
> > > 
> > > -Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\
**Post date:** [February 25, 2013, 6:31pm UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/5 "2013-02-25T18:31:30Z")

</div>

do you have the index that is corrupted still? if so I really would love to  
look at it if possible? can you provide it somehow. Contact me at  
[simon.willnauer@elasticsearch.com](mailto:simon.willnauer@elasticsearch.com)

thanks!!

simon

On Monday, February 25, 2013 7:10:18 PM UTC+1, Taras Shkvarchuk wrote:

> Not sure what type of info you're looking for, so you can e-mail me for  
> more details, I can probably even provide the index in question for  
> debugging.  
> There are 420K root level documents as all of which have small nested  
> documents. Re-indexing nearly the same data resulted in an index without  
> corruption.
> 
> Here is the \_status and \_segments call.  
> {  
> "ok" : true,  
> "\_shards" : {  
> "total" : 1,  
> "successful" : 1,  
> "failed" : 0  
> },  
> "indices" : {  
> "events" : {  
> "index" : {  
> "primary\_size" : "673.3mb",  
> "primary\_size\_in\_bytes" : 706043580,  
> "size" : "673.3mb",  
> "size\_in\_bytes" : 706043580  
> },  
> "translog" : {  
> "operations" : 0  
> },  
> "docs" : {  
> "num\_docs" : 1869131,  
> "max\_doc" : 2263281,  
> "deleted\_docs" : 394150  
> },  
> "merges" : {  
> "current" : 0,  
> "current\_docs" : 0,  
> "current\_size" : "0b",  
> "current\_size\_in\_bytes" : 0,  
> "total" : 1,  
> "total\_time" : "3.7s",  
> "total\_time\_in\_millis" : 3776,  
> "total\_docs" : 2263281,  
> "total\_size" : "0b",  
> "total\_size\_in\_bytes" : 0  
> },  
> "refresh" : {  
> "total" : 1,  
> "total\_time" : "0s",  
> "total\_time\_in\_millis" : 0  
> },  
> "flush" : {  
> "total" : 1,  
> "total\_time" : "18ms",  
> "total\_time\_in\_millis" : 18  
> },  
> "shards" : {  
> "0" : [ {  
> "routing" : {  
> "state" : "STARTED",  
> "primary" : true,  
> "node" : "vwbSeFjtQ3SqdbJzh8V-uA",  
> "relocating\_node" : null,  
> "shard" : 0,  
> "index" : "events"  
> },  
> "state" : "STARTED",  
> "index" : {  
> "size" : "673.3mb",  
> "size\_in\_bytes" : 706043580  
> },  
> "translog" : {  
> "id" : 1361432592857,  
> "operations" : 0  
> },  
> "docs" : {  
> "num\_docs" : 1869131,  
> "max\_doc" : 2263281,  
> "deleted\_docs" : 394150  
> },  
> "merges" : {  
> "current" : 0,  
> "current\_docs" : 0,  
> "current\_size" : "0b",  
> "current\_size\_in\_bytes" : 0,  
> "total" : 1,  
> "total\_time" : "3.7s",  
> "total\_time\_in\_millis" : 3776,  
> "total\_docs" : 2263281,  
> "total\_size" : "0b",  
> "total\_size\_in\_bytes" : 0  
> },  
> "refresh" : {  
> "total" : 1,  
> "total\_time" : "0s",  
> "total\_time\_in\_millis" : 0  
> },  
> "flush" : {  
> "total" : 1,  
> "total\_time" : "18ms",  
> "total\_time\_in\_millis" : 18  
> }  
> } ]  
> }  
> }  
> }
> 
> \_segments
> 
> {  
> "ok" : true,  
> "\_shards" : {  
> "total" : 1,  
> "successful" : 1,  
> "failed" : 0  
> },  
> "indices" : {  
> "events" : {  
> "shards" : {  
> "0" : [ {  
> "routing" : {  
> "state" : "STARTED",  
> "primary" : true,  
> "node" : "vwbSeFjtQ3SqdbJzh8V-uA"  
> },  
> "num\_committed\_segments" : 20,  
> "num\_search\_segments" : 20,  
> "segments" : {  
> "\_4qm" : {  
> "generation" : 6142,  
> "num\_docs" : 6730,  
> "deleted\_docs" : 67313,  
> "size" : "29.8mb",  
> "size\_in\_bytes" : 31342776,  
> "committed" : true,  
> "search" : true  
> },  
> "\_ukk" : {  
> "generation" : 39620,  
> "num\_docs" : 579895,  
> "deleted\_docs" : 232003,  
> "size" : "203.9mb",  
> "size\_in\_bytes" : 213859775,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1o32" : {  
> "generation" : 77870,  
> "num\_docs" : 1131181,  
> "deleted\_docs" : 71964,  
> "size" : "389.9mb",  
> "size\_in\_bytes" : 408880404,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1o3e" : {  
> "generation" : 77882,  
> "num\_docs" : 17179,  
> "deleted\_docs" : 15495,  
> "size" : "9.9mb",  
> "size\_in\_bytes" : 10401244,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1qse" : {  
> "generation" : 81374,  
> "num\_docs" : 64627,  
> "deleted\_docs" : 1870,  
> "size" : "16.5mb",  
> "size\_in\_bytes" : 17305817,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1qyf" : {  
> "generation" : 81591,  
> "num\_docs" : 9475,  
> "deleted\_docs" : 2073,  
> "size" : "3.7mb",  
> "size\_in\_bytes" : 3894229,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1r7a" : {  
> "generation" : 81910,  
> "num\_docs" : 15972,  
> "deleted\_docs" : 2375,  
> "size" : "5mb",  
> "size\_in\_bytes" : 5267040,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rck" : {  
> "generation" : 82100,  
> "num\_docs" : 9818,  
> "deleted\_docs" : 6,  
> "size" : "3.5mb",  
> "size\_in\_bytes" : 3753253,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1re0" : {  
> "generation" : 82152,  
> "num\_docs" : 16651,  
> "deleted\_docs" : 787,  
> "size" : "4.6mb",  
> "size\_in\_bytes" : 4896408,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rhz" : {  
> "generation" : 82295,  
> "num\_docs" : 14006,  
> "deleted\_docs" : 264,  
> "size" : "4.7mb",  
> "size\_in\_bytes" : 5016838,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri0" : {  
> "generation" : 82296,  
> "num\_docs" : 479,  
> "deleted\_docs" : 0,  
> "size" : "162.7kb",  
> "size\_in\_bytes" : 166651,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri4" : {  
> "generation" : 82300,  
> "num\_docs" : 266,  
> "deleted\_docs" : 0,  
> "size" : "110.1kb",  
> "size\_in\_bytes" : 112763,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri6" : {  
> "generation" : 82302,  
> "num\_docs" : 250,  
> "deleted\_docs" : 0,  
> "size" : "109.3kb",  
> "size\_in\_bytes" : 111965,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri8" : {  
> "generation" : 82304,  
> "num\_docs" : 153,  
> "deleted\_docs" : 0,  
> "size" : "85.4kb",  
> "size\_in\_bytes" : 87460,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ria" : {  
> "generation" : 82306,  
> "num\_docs" : 549,  
> "deleted\_docs" : 0,  
> "size" : "190.7kb",  
> "size\_in\_bytes" : 195340,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1riu" : {  
> "generation" : 82326,  
> "num\_docs" : 261,  
> "deleted\_docs" : 0,  
> "size" : "106kb",  
> "size\_in\_bytes" : 108604,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rj6" : {  
> "generation" : 82338,  
> "num\_docs" : 533,  
> "deleted\_docs" : 0,  
> "size" : "191.1kb",  
> "size\_in\_bytes" : 195748,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rj8" : {  
> "generation" : 82340,  
> "num\_docs" : 276,  
> "deleted\_docs" : 0,  
> "size" : "119.4kb",  
> "size\_in\_bytes" : 122299,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rjb" : {  
> "generation" : 82343,  
> "num\_docs" : 251,  
> "deleted\_docs" : 0,  
> "size" : "100.7kb",  
> "size\_in\_bytes" : 103129,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rjs" : {  
> "generation" : 82360,  
> "num\_docs" : 579,  
> "deleted\_docs" : 0,  
> "size" : "207.2kb",  
> "size\_in\_bytes" : 212180,  
> "committed" : true,  
> "search" : true  
> }  
> }  
> } ]  
> }  
> }  
> }  
> }
> 
> On Monday, February 25, 2013 6:22:31 AM UTC-8, simonw wrote:
> 
> > Hey Taras,
> > 
> > it would be awesome if you could provide some more infos about you  
> > problems so we can fix it quickly
> > 
> > simon
> > 
> > On Monday, February 25, 2013 9:42:20 AM UTC+1, simonw wrote:
> > 
> > > hey,
> > > 
> > > can you give some more background here, like did you start this on an  
> > > old index ie. an index you created with a previous version of ES / Lucene?  
> > > Can you reproduce this?
> > > 
> > > simon
> > > 
> > > On Monday, February 25, 2013 7:54:46 AM UTC+1, Taras Shkvarchuk wrote:
> > > 
> > > > While testing the master build with lucene 4.1, I got a permanent merge  
> > > > error filling up my logs.  
> > > > Has anyone else seen it? Anything I can provide to help debug?
> > > > 
> > > > [2013-02-24 22:43:38,863][WARN][index.merge.scheduler]  
> > > > [Enchantress] [events][0] failed to merge  
> > > > java.lang.ArrayIndexOutOfBoundsException  
> > > > at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
> > > > at  
> > > > org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
> > > > at  
> > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
> > > > at  
> > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
> > > > at  
> > > > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
> > > > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
> > > > at  
> > > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
> > > > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
> > > > at  
> > > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
> > > > at  
> > > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
> > > > at  
> > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
> > > > 
> > > > Thanks!
> > > > 
> > > > -Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\
**Post date:** [February 25, 2013, 6:34pm UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/6 "2013-02-25T18:34:41Z")

</div>

one thing that would be interesting is what JVM are you using, is there a  
possibility for a JVM bug here?

simon

On Monday, February 25, 2013 7:10:18 PM UTC+1, Taras Shkvarchuk wrote:

> Not sure what type of info you're looking for, so you can e-mail me for  
> more details, I can probably even provide the index in question for  
> debugging.  
> There are 420K root level documents as all of which have small nested  
> documents. Re-indexing nearly the same data resulted in an index without  
> corruption.
> 
> Here is the \_status and \_segments call.  
> {  
> "ok" : true,  
> "\_shards" : {  
> "total" : 1,  
> "successful" : 1,  
> "failed" : 0  
> },  
> "indices" : {  
> "events" : {  
> "index" : {  
> "primary\_size" : "673.3mb",  
> "primary\_size\_in\_bytes" : 706043580,  
> "size" : "673.3mb",  
> "size\_in\_bytes" : 706043580  
> },  
> "translog" : {  
> "operations" : 0  
> },  
> "docs" : {  
> "num\_docs" : 1869131,  
> "max\_doc" : 2263281,  
> "deleted\_docs" : 394150  
> },  
> "merges" : {  
> "current" : 0,  
> "current\_docs" : 0,  
> "current\_size" : "0b",  
> "current\_size\_in\_bytes" : 0,  
> "total" : 1,  
> "total\_time" : "3.7s",  
> "total\_time\_in\_millis" : 3776,  
> "total\_docs" : 2263281,  
> "total\_size" : "0b",  
> "total\_size\_in\_bytes" : 0  
> },  
> "refresh" : {  
> "total" : 1,  
> "total\_time" : "0s",  
> "total\_time\_in\_millis" : 0  
> },  
> "flush" : {  
> "total" : 1,  
> "total\_time" : "18ms",  
> "total\_time\_in\_millis" : 18  
> },  
> "shards" : {  
> "0" : [ {  
> "routing" : {  
> "state" : "STARTED",  
> "primary" : true,  
> "node" : "vwbSeFjtQ3SqdbJzh8V-uA",  
> "relocating\_node" : null,  
> "shard" : 0,  
> "index" : "events"  
> },  
> "state" : "STARTED",  
> "index" : {  
> "size" : "673.3mb",  
> "size\_in\_bytes" : 706043580  
> },  
> "translog" : {  
> "id" : 1361432592857,  
> "operations" : 0  
> },  
> "docs" : {  
> "num\_docs" : 1869131,  
> "max\_doc" : 2263281,  
> "deleted\_docs" : 394150  
> },  
> "merges" : {  
> "current" : 0,  
> "current\_docs" : 0,  
> "current\_size" : "0b",  
> "current\_size\_in\_bytes" : 0,  
> "total" : 1,  
> "total\_time" : "3.7s",  
> "total\_time\_in\_millis" : 3776,  
> "total\_docs" : 2263281,  
> "total\_size" : "0b",  
> "total\_size\_in\_bytes" : 0  
> },  
> "refresh" : {  
> "total" : 1,  
> "total\_time" : "0s",  
> "total\_time\_in\_millis" : 0  
> },  
> "flush" : {  
> "total" : 1,  
> "total\_time" : "18ms",  
> "total\_time\_in\_millis" : 18  
> }  
> } ]  
> }  
> }  
> }
> 
> \_segments
> 
> {  
> "ok" : true,  
> "\_shards" : {  
> "total" : 1,  
> "successful" : 1,  
> "failed" : 0  
> },  
> "indices" : {  
> "events" : {  
> "shards" : {  
> "0" : [ {  
> "routing" : {  
> "state" : "STARTED",  
> "primary" : true,  
> "node" : "vwbSeFjtQ3SqdbJzh8V-uA"  
> },  
> "num\_committed\_segments" : 20,  
> "num\_search\_segments" : 20,  
> "segments" : {  
> "\_4qm" : {  
> "generation" : 6142,  
> "num\_docs" : 6730,  
> "deleted\_docs" : 67313,  
> "size" : "29.8mb",  
> "size\_in\_bytes" : 31342776,  
> "committed" : true,  
> "search" : true  
> },  
> "\_ukk" : {  
> "generation" : 39620,  
> "num\_docs" : 579895,  
> "deleted\_docs" : 232003,  
> "size" : "203.9mb",  
> "size\_in\_bytes" : 213859775,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1o32" : {  
> "generation" : 77870,  
> "num\_docs" : 1131181,  
> "deleted\_docs" : 71964,  
> "size" : "389.9mb",  
> "size\_in\_bytes" : 408880404,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1o3e" : {  
> "generation" : 77882,  
> "num\_docs" : 17179,  
> "deleted\_docs" : 15495,  
> "size" : "9.9mb",  
> "size\_in\_bytes" : 10401244,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1qse" : {  
> "generation" : 81374,  
> "num\_docs" : 64627,  
> "deleted\_docs" : 1870,  
> "size" : "16.5mb",  
> "size\_in\_bytes" : 17305817,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1qyf" : {  
> "generation" : 81591,  
> "num\_docs" : 9475,  
> "deleted\_docs" : 2073,  
> "size" : "3.7mb",  
> "size\_in\_bytes" : 3894229,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1r7a" : {  
> "generation" : 81910,  
> "num\_docs" : 15972,  
> "deleted\_docs" : 2375,  
> "size" : "5mb",  
> "size\_in\_bytes" : 5267040,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rck" : {  
> "generation" : 82100,  
> "num\_docs" : 9818,  
> "deleted\_docs" : 6,  
> "size" : "3.5mb",  
> "size\_in\_bytes" : 3753253,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1re0" : {  
> "generation" : 82152,  
> "num\_docs" : 16651,  
> "deleted\_docs" : 787,  
> "size" : "4.6mb",  
> "size\_in\_bytes" : 4896408,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rhz" : {  
> "generation" : 82295,  
> "num\_docs" : 14006,  
> "deleted\_docs" : 264,  
> "size" : "4.7mb",  
> "size\_in\_bytes" : 5016838,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri0" : {  
> "generation" : 82296,  
> "num\_docs" : 479,  
> "deleted\_docs" : 0,  
> "size" : "162.7kb",  
> "size\_in\_bytes" : 166651,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri4" : {  
> "generation" : 82300,  
> "num\_docs" : 266,  
> "deleted\_docs" : 0,  
> "size" : "110.1kb",  
> "size\_in\_bytes" : 112763,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri6" : {  
> "generation" : 82302,  
> "num\_docs" : 250,  
> "deleted\_docs" : 0,  
> "size" : "109.3kb",  
> "size\_in\_bytes" : 111965,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ri8" : {  
> "generation" : 82304,  
> "num\_docs" : 153,  
> "deleted\_docs" : 0,  
> "size" : "85.4kb",  
> "size\_in\_bytes" : 87460,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1ria" : {  
> "generation" : 82306,  
> "num\_docs" : 549,  
> "deleted\_docs" : 0,  
> "size" : "190.7kb",  
> "size\_in\_bytes" : 195340,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1riu" : {  
> "generation" : 82326,  
> "num\_docs" : 261,  
> "deleted\_docs" : 0,  
> "size" : "106kb",  
> "size\_in\_bytes" : 108604,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rj6" : {  
> "generation" : 82338,  
> "num\_docs" : 533,  
> "deleted\_docs" : 0,  
> "size" : "191.1kb",  
> "size\_in\_bytes" : 195748,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rj8" : {  
> "generation" : 82340,  
> "num\_docs" : 276,  
> "deleted\_docs" : 0,  
> "size" : "119.4kb",  
> "size\_in\_bytes" : 122299,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rjb" : {  
> "generation" : 82343,  
> "num\_docs" : 251,  
> "deleted\_docs" : 0,  
> "size" : "100.7kb",  
> "size\_in\_bytes" : 103129,  
> "committed" : true,  
> "search" : true  
> },  
> "\_1rjs" : {  
> "generation" : 82360,  
> "num\_docs" : 579,  
> "deleted\_docs" : 0,  
> "size" : "207.2kb",  
> "size\_in\_bytes" : 212180,  
> "committed" : true,  
> "search" : true  
> }  
> }  
> } ]  
> }  
> }  
> }  
> }
> 
> On Monday, February 25, 2013 6:22:31 AM UTC-8, simonw wrote:
> 
> > Hey Taras,
> > 
> > it would be awesome if you could provide some more infos about you  
> > problems so we can fix it quickly
> > 
> > simon
> > 
> > On Monday, February 25, 2013 9:42:20 AM UTC+1, simonw wrote:
> > 
> > > hey,
> > > 
> > > can you give some more background here, like did you start this on an  
> > > old index ie. an index you created with a previous version of ES / Lucene?  
> > > Can you reproduce this?
> > > 
> > > simon
> > > 
> > > On Monday, February 25, 2013 7:54:46 AM UTC+1, Taras Shkvarchuk wrote:
> > > 
> > > > While testing the master build with lucene 4.1, I got a permanent merge  
> > > > error filling up my logs.  
> > > > Has anyone else seen it? Anything I can provide to help debug?
> > > > 
> > > > [2013-02-24 22:43:38,863][WARN][index.merge.scheduler]  
> > > > [Enchantress] [events][0] failed to merge  
> > > > java.lang.ArrayIndexOutOfBoundsException  
> > > > at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
> > > > at  
> > > > org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
> > > > at  
> > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
> > > > at  
> > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
> > > > at  
> > > > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
> > > > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
> > > > at  
> > > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
> > > > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
> > > > at  
> > > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
> > > > at  
> > > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
> > > > at  
> > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
> > > > 
> > > > Thanks!
> > > > 
> > > > -Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Simon\_Effenberg](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@Simon\_Effenberg](https://discuss.elastic.co/u/Simon_Effenberg)\
**Post date:** [July 24, 2013, 6:47am UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/7 "2013-07-24T06:47:14Z")

</div>

Hi,

we have had the same problem on the weekend.

[2013-07-19 14:53:16,663][WARN][index.merge.scheduler] [  
kreplytssearch47-1] [replyts][7] failed to merge  
java.lang.ArrayIndexOutOfBoundsException: 413166  
at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:  
146)  
at org.apache.lucene.index.MergeState$DocMap$1.get(MergeState.java:  
86)  
at org.apache.lucene.codecs.MappingMultiDocsAndPositionsEnum.nextDoc  
(MappingMultiDocsAndPositionsEnum.java:107)  
at org.apache.lucene.codecs.PostingsConsumer.merge(PostingsConsumer.  
java:109)  
at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:  
164)  
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java  
:72)  
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.  
java:365)  
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:98  
)  
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:  
3709)  
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3313)  
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(  
ConcurrentMergeScheduler.java:401)  
at org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(  
TrackingConcurrentMergeScheduler.java:91)  
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(  
ConcurrentMergeScheduler.java:478)  
[2013-07-19 14:53:16,731][WARN][index.engine.robin] [  
kreplytssearch47-1] [replyts][7] failed engine  
org.apache.lucene.index.MergePolicy$MergeException: java.lang.  
ArrayIndexOutOfBoundsException: 413166  
at org.elasticsearch.index.merge.scheduler.  
ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.  
handleMergeException(ConcurrentMergeSchedulerProvider.java:100)  
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(  
ConcurrentMergeScheduler.java:514)  
Caused by: java.lang.ArrayIndexOutOfBoundsException: 413166  
at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:  
146)  
at org.apache.lucene.index.MergeState$DocMap$1.get(MergeState.java:  
86)  
at org.apache.lucene.codecs.MappingMultiDocsAndPositionsEnum.nextDoc  
(MappingMultiDocsAndPositionsEnum.java:107)  
at org.apache.lucene.codecs.PostingsConsumer.merge(PostingsConsumer.  
java:109)  
at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:  
164)  
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java  
:72)  
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.  
java:365)  
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:98  
)  
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:  
3709)  
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3313)  
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(  
ConcurrentMergeScheduler.java:401)  
at org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(  
TrackingConcurrentMergeScheduler.java:91)  
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(  
ConcurrentMergeScheduler.java:478)  
[2013-07-19 14:53:16,790][WARN][cluster.action.shard] [  
kreplytssearch47-1] sending failed shard for [replyts][7], node[  
lq6vXWmaSlyNFAagtRu6uQ], [R], s[STARTED], reason [engine failure, message [  
MergeException[java.lang.ArrayIndexOutOfBoundsException: 413166]; nested:  
ArrayIndexOutOfBoundsException[413166]; ]]

The cluster is actually a 0.90.1 with JVM

java version "1.7.0\_21"  
Java(TM) SE Runtime Environment (build 1.7.0\_21-b11)  
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

the index was previously created with 0.20.2 and then only the software was  
upgraded.

I have had to remove the single shard and replaced it with an empty version  
to be able to recover.

One important thing to add: after the exception happened the cluster  
started to behave bad:

- node A tried to send the failed shard to node B
- while recovering on B the recovering failed with this exception
- B was somehow renaming the broken files
- A tried to send the failed shard again to node B
- while recovering it failed again...
- B renamed the files again...  
....  
...

this didn't stop until the disk was full on node B.

Cheers  
Simon

PS: Hi Simon.. so "trifft" man sich wieder ;).. hast du noch Kontakt zu den  
alten TFH Profs?

On Monday, February 25, 2013 7:34:41 PM UTC+1, simonw wrote:

> one thing that would be interesting is what JVM are you using, is there a  
> possibility for a JVM bug here?
> 
> simon
> 
> On Monday, February 25, 2013 7:10:18 PM UTC+1, Taras Shkvarchuk wrote:
> 
> > Not sure what type of info you're looking for, so you can e-mail me for  
> > more details, I can probably even provide the index in question for  
> > debugging.  
> > There are 420K root level documents as all of which have small nested  
> > documents. Re-indexing nearly the same data resulted in an index without  
> > corruption.
> > 
> > Here is the \_status and \_segments call.  
> > {  
> > "ok" : true,  
> > "\_shards" : {  
> > "total" : 1,  
> > "successful" : 1,  
> > "failed" : 0  
> > },  
> > "indices" : {  
> > "events" : {  
> > "index" : {  
> > "primary\_size" : "673.3mb",  
> > "primary\_size\_in\_bytes" : 706043580,  
> > "size" : "673.3mb",  
> > "size\_in\_bytes" : 706043580  
> > },  
> > "translog" : {  
> > "operations" : 0  
> > },  
> > "docs" : {  
> > "num\_docs" : 1869131,  
> > "max\_doc" : 2263281,  
> > "deleted\_docs" : 394150  
> > },  
> > "merges" : {  
> > "current" : 0,  
> > "current\_docs" : 0,  
> > "current\_size" : "0b",  
> > "current\_size\_in\_bytes" : 0,  
> > "total" : 1,  
> > "total\_time" : "3.7s",  
> > "total\_time\_in\_millis" : 3776,  
> > "total\_docs" : 2263281,  
> > "total\_size" : "0b",  
> > "total\_size\_in\_bytes" : 0  
> > },  
> > "refresh" : {  
> > "total" : 1,  
> > "total\_time" : "0s",  
> > "total\_time\_in\_millis" : 0  
> > },  
> > "flush" : {  
> > "total" : 1,  
> > "total\_time" : "18ms",  
> > "total\_time\_in\_millis" : 18  
> > },  
> > "shards" : {  
> > "0" : [ {  
> > "routing" : {  
> > "state" : "STARTED",  
> > "primary" : true,  
> > "node" : "vwbSeFjtQ3SqdbJzh8V-uA",  
> > "relocating\_node" : null,  
> > "shard" : 0,  
> > "index" : "events"  
> > },  
> > "state" : "STARTED",  
> > "index" : {  
> > "size" : "673.3mb",  
> > "size\_in\_bytes" : 706043580  
> > },  
> > "translog" : {  
> > "id" : 1361432592857,  
> > "operations" : 0  
> > },  
> > "docs" : {  
> > "num\_docs" : 1869131,  
> > "max\_doc" : 2263281,  
> > "deleted\_docs" : 394150  
> > },  
> > "merges" : {  
> > "current" : 0,  
> > "current\_docs" : 0,  
> > "current\_size" : "0b",  
> > "current\_size\_in\_bytes" : 0,  
> > "total" : 1,  
> > "total\_time" : "3.7s",  
> > "total\_time\_in\_millis" : 3776,  
> > "total\_docs" : 2263281,  
> > "total\_size" : "0b",  
> > "total\_size\_in\_bytes" : 0  
> > },  
> > "refresh" : {  
> > "total" : 1,  
> > "total\_time" : "0s",  
> > "total\_time\_in\_millis" : 0  
> > },  
> > "flush" : {  
> > "total" : 1,  
> > "total\_time" : "18ms",  
> > "total\_time\_in\_millis" : 18  
> > }  
> > } ]  
> > }  
> > }  
> > }
> > 
> > \_segments
> > 
> > {  
> > "ok" : true,  
> > "\_shards" : {  
> > "total" : 1,  
> > "successful" : 1,  
> > "failed" : 0  
> > },  
> > "indices" : {  
> > "events" : {  
> > "shards" : {  
> > "0" : [ {  
> > "routing" : {  
> > "state" : "STARTED",  
> > "primary" : true,  
> > "node" : "vwbSeFjtQ3SqdbJzh8V-uA"  
> > },  
> > "num\_committed\_segments" : 20,  
> > "num\_search\_segments" : 20,  
> > "segments" : {  
> > "\_4qm" : {  
> > "generation" : 6142,  
> > "num\_docs" : 6730,  
> > "deleted\_docs" : 67313,  
> > "size" : "29.8mb",  
> > "size\_in\_bytes" : 31342776,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_ukk" : {  
> > "generation" : 39620,  
> > "num\_docs" : 579895,  
> > "deleted\_docs" : 232003,  
> > "size" : "203.9mb",  
> > "size\_in\_bytes" : 213859775,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1o32" : {  
> > "generation" : 77870,  
> > "num\_docs" : 1131181,  
> > "deleted\_docs" : 71964,  
> > "size" : "389.9mb",  
> > "size\_in\_bytes" : 408880404,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1o3e" : {  
> > "generation" : 77882,  
> > "num\_docs" : 17179,  
> > "deleted\_docs" : 15495,  
> > "size" : "9.9mb",  
> > "size\_in\_bytes" : 10401244,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1qse" : {  
> > "generation" : 81374,  
> > "num\_docs" : 64627,  
> > "deleted\_docs" : 1870,  
> > "size" : "16.5mb",  
> > "size\_in\_bytes" : 17305817,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1qyf" : {  
> > "generation" : 81591,  
> > "num\_docs" : 9475,  
> > "deleted\_docs" : 2073,  
> > "size" : "3.7mb",  
> > "size\_in\_bytes" : 3894229,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1r7a" : {  
> > "generation" : 81910,  
> > "num\_docs" : 15972,  
> > "deleted\_docs" : 2375,  
> > "size" : "5mb",  
> > "size\_in\_bytes" : 5267040,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1rck" : {  
> > "generation" : 82100,  
> > "num\_docs" : 9818,  
> > "deleted\_docs" : 6,  
> > "size" : "3.5mb",  
> > "size\_in\_bytes" : 3753253,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1re0" : {  
> > "generation" : 82152,  
> > "num\_docs" : 16651,  
> > "deleted\_docs" : 787,  
> > "size" : "4.6mb",  
> > "size\_in\_bytes" : 4896408,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1rhz" : {  
> > "generation" : 82295,  
> > "num\_docs" : 14006,  
> > "deleted\_docs" : 264,  
> > "size" : "4.7mb",  
> > "size\_in\_bytes" : 5016838,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1ri0" : {  
> > "generation" : 82296,  
> > "num\_docs" : 479,  
> > "deleted\_docs" : 0,  
> > "size" : "162.7kb",  
> > "size\_in\_bytes" : 166651,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1ri4" : {  
> > "generation" : 82300,  
> > "num\_docs" : 266,  
> > "deleted\_docs" : 0,  
> > "size" : "110.1kb",  
> > "size\_in\_bytes" : 112763,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1ri6" : {  
> > "generation" : 82302,  
> > "num\_docs" : 250,  
> > "deleted\_docs" : 0,  
> > "size" : "109.3kb",  
> > "size\_in\_bytes" : 111965,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1ri8" : {  
> > "generation" : 82304,  
> > "num\_docs" : 153,  
> > "deleted\_docs" : 0,  
> > "size" : "85.4kb",  
> > "size\_in\_bytes" : 87460,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1ria" : {  
> > "generation" : 82306,  
> > "num\_docs" : 549,  
> > "deleted\_docs" : 0,  
> > "size" : "190.7kb",  
> > "size\_in\_bytes" : 195340,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1riu" : {  
> > "generation" : 82326,  
> > "num\_docs" : 261,  
> > "deleted\_docs" : 0,  
> > "size" : "106kb",  
> > "size\_in\_bytes" : 108604,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1rj6" : {  
> > "generation" : 82338,  
> > "num\_docs" : 533,  
> > "deleted\_docs" : 0,  
> > "size" : "191.1kb",  
> > "size\_in\_bytes" : 195748,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1rj8" : {  
> > "generation" : 82340,  
> > "num\_docs" : 276,  
> > "deleted\_docs" : 0,  
> > "size" : "119.4kb",  
> > "size\_in\_bytes" : 122299,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1rjb" : {  
> > "generation" : 82343,  
> > "num\_docs" : 251,  
> > "deleted\_docs" : 0,  
> > "size" : "100.7kb",  
> > "size\_in\_bytes" : 103129,  
> > "committed" : true,  
> > "search" : true  
> > },  
> > "\_1rjs" : {  
> > "generation" : 82360,  
> > "num\_docs" : 579,  
> > "deleted\_docs" : 0,  
> > "size" : "207.2kb",  
> > "size\_in\_bytes" : 212180,  
> > "committed" : true,  
> > "search" : true  
> > }  
> > }  
> > } ]  
> > }  
> > }  
> > }  
> > }
> > 
> > On Monday, February 25, 2013 6:22:31 AM UTC-8, simonw wrote:
> > 
> > > Hey Taras,
> > > 
> > > it would be awesome if you could provide some more infos about you  
> > > problems so we can fix it quickly
> > > 
> > > simon
> > > 
> > > On Monday, February 25, 2013 9:42:20 AM UTC+1, simonw wrote:
> > > 
> > > > hey,
> > > > 
> > > > can you give some more background here, like did you start this on an  
> > > > old index ie. an index you created with a previous version of ES / Lucene?  
> > > > Can you reproduce this?
> > > > 
> > > > simon
> > > > 
> > > > On Monday, February 25, 2013 7:54:46 AM UTC+1, Taras Shkvarchuk wrote:
> > > > 
> > > > > While testing the master build with lucene 4.1, I got a permanent  
> > > > > merge error filling up my logs.  
> > > > > Has anyone else seen it? Anything I can provide to help debug?
> > > > > 
> > > > > [2013-02-24 22:43:38,863][WARN][index.merge.scheduler]  
> > > > > [Enchantress] [events][0] failed to merge  
> > > > > java.lang.ArrayIndexOutOfBoundsException  
> > > > > at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
> > > > > at  
> > > > > org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
> > > > > at  
> > > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
> > > > > at  
> > > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
> > > > > at  
> > > > > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
> > > > > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
> > > > > at  
> > > > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
> > > > > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
> > > > > at  
> > > > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
> > > > > at  
> > > > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
> > > > > at  
> > > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
> > > > > 
> > > > > Thanks!
> > > > > 
> > > > > -Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\
**Post date:** [July 29, 2013, 6:58pm UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/8 "2013-07-29T18:58:18Z")

</div>

hey simon!

good to meet you again, long time no see! Is there any chance we can get a  
hold on this corrupted shard? It would be awesome to get this and reproduce  
the problem somehow. You can contact me on  
[simon.willnauer@elasticsearch.com](mailto:simon.willnauer@elasticsearch.com) directly though!

R u still around in Berlin, wanna meet for a coffee at some point?

simon

On Wednesday, July 24, 2013 8:47:14 AM UTC+2, Simon Effenberg wrote:

> Hi,
> 
> we have had the same problem on the weekend.
> 
> [2013-07-19 14:53:16,663][WARN][index.merge.scheduler] [  
> kreplytssearch47-1] [replyts][7] failed to merge  
> java.lang.ArrayIndexOutOfBoundsException: 413166  
> at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:  
> 146)  
> at org.apache.lucene.index.MergeState$DocMap$1.get(MergeState.java  
> :86)  
> at org.apache.lucene.codecs.MappingMultiDocsAndPositionsEnum.  
> nextDoc(MappingMultiDocsAndPositionsEnum.java:107)  
> at org.apache.lucene.codecs.PostingsConsumer.merge(  
> PostingsConsumer.java:109)  
> at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java  
> :164)  
> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.  
> java:72)  
> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.  
> java:365)  
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:  
> 98)  
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.  
> java:3709)  
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3313  
> )  
> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(  
> ConcurrentMergeScheduler.java:401)  
> at org.apache.lucene.index.TrackingConcurrentMergeScheduler.  
> doMerge(TrackingConcurrentMergeScheduler.java:91)  
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.  
> run(ConcurrentMergeScheduler.java:478)  
> [2013-07-19 14:53:16,731][WARN][index.engine.robin] [  
> kreplytssearch47-1] [replyts][7] failed engine  
> org.apache.lucene.index.MergePolicy$MergeException: java.lang.  
> ArrayIndexOutOfBoundsException: 413166  
> at org.elasticsearch.index.merge.scheduler.  
> ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.  
> handleMergeException(ConcurrentMergeSchedulerProvider.java:100)  
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.  
> run(ConcurrentMergeScheduler.java:514)  
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 413166  
> at org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:  
> 146)  
> at org.apache.lucene.index.MergeState$DocMap$1.get(MergeState.java  
> :86)  
> at org.apache.lucene.codecs.MappingMultiDocsAndPositionsEnum.  
> nextDoc(MappingMultiDocsAndPositionsEnum.java:107)  
> at org.apache.lucene.codecs.PostingsConsumer.merge(  
> PostingsConsumer.java:109)  
> at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java  
> :164)  
> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.  
> java:72)  
> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.  
> java:365)  
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:  
> 98)  
> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.  
> java:3709)  
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3313  
> )  
> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(  
> ConcurrentMergeScheduler.java:401)  
> at org.apache.lucene.index.TrackingConcurrentMergeScheduler.  
> doMerge(TrackingConcurrentMergeScheduler.java:91)  
> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.  
> run(ConcurrentMergeScheduler.java:478)  
> [2013-07-19 14:53:16,790][WARN][cluster.action.shard] [  
> kreplytssearch47-1] sending failed shard for [replyts][7], node[  
> lq6vXWmaSlyNFAagtRu6uQ], [R], s[STARTED], reason [engine failure, message  
> [MergeException[java.lang.ArrayIndexOutOfBoundsException: 413166]; nested:  
> ArrayIndexOutOfBoundsException[413166]; ]]
> 
> The cluster is actually a 0.90.1 with JVM
> 
> java version "1.7.0\_21"  
> Java(TM) SE Runtime Environment (build 1.7.0\_21-b11)  
> Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)
> 
> the index was previously created with 0.20.2 and then only the software  
> was upgraded.
> 
> I have had to remove the single shard and replaced it with an empty  
> version to be able to recover.
> 
> One important thing to add: after the exception happened the cluster  
> started to behave bad:
> 
> - node A tried to send the failed shard to node B
> - while recovering on B the recovering failed with this exception
> - B was somehow renaming the broken files
> - A tried to send the failed shard again to node B
> - while recovering it failed again...
> - B renamed the files again...  
> ....  
> ...
> 
> this didn't stop until the disk was full on node B.
> 
> Cheers  
> Simon
> 
> PS: Hi Simon.. so "trifft" man sich wieder ;).. hast du noch Kontakt zu  
> den alten TFH Profs?
> 
> On Monday, February 25, 2013 7:34:41 PM UTC+1, simonw wrote:
> 
> > one thing that would be interesting is what JVM are you using, is there a  
> > possibility for a JVM bug here?
> > 
> > simon
> > 
> > On Monday, February 25, 2013 7:10:18 PM UTC+1, Taras Shkvarchuk wrote:
> > 
> > > Not sure what type of info you're looking for, so you can e-mail me for  
> > > more details, I can probably even provide the index in question for  
> > > debugging.  
> > > There are 420K root level documents as all of which have small nested  
> > > documents. Re-indexing nearly the same data resulted in an index without  
> > > corruption.
> > > 
> > > Here is the \_status and \_segments call.  
> > > {  
> > > "ok" : true,  
> > > "\_shards" : {  
> > > "total" : 1,  
> > > "successful" : 1,  
> > > "failed" : 0  
> > > },  
> > > "indices" : {  
> > > "events" : {  
> > > "index" : {  
> > > "primary\_size" : "673.3mb",  
> > > "primary\_size\_in\_bytes" : 706043580,  
> > > "size" : "673.3mb",  
> > > "size\_in\_bytes" : 706043580  
> > > },  
> > > "translog" : {  
> > > "operations" : 0  
> > > },  
> > > "docs" : {  
> > > "num\_docs" : 1869131,  
> > > "max\_doc" : 2263281,  
> > > "deleted\_docs" : 394150  
> > > },  
> > > "merges" : {  
> > > "current" : 0,  
> > > "current\_docs" : 0,  
> > > "current\_size" : "0b",  
> > > "current\_size\_in\_bytes" : 0,  
> > > "total" : 1,  
> > > "total\_time" : "3.7s",  
> > > "total\_time\_in\_millis" : 3776,  
> > > "total\_docs" : 2263281,  
> > > "total\_size" : "0b",  
> > > "total\_size\_in\_bytes" : 0  
> > > },  
> > > "refresh" : {  
> > > "total" : 1,  
> > > "total\_time" : "0s",  
> > > "total\_time\_in\_millis" : 0  
> > > },  
> > > "flush" : {  
> > > "total" : 1,  
> > > "total\_time" : "18ms",  
> > > "total\_time\_in\_millis" : 18  
> > > },  
> > > "shards" : {  
> > > "0" : [ {  
> > > "routing" : {  
> > > "state" : "STARTED",  
> > > "primary" : true,  
> > > "node" : "vwbSeFjtQ3SqdbJzh8V-uA",  
> > > "relocating\_node" : null,  
> > > "shard" : 0,  
> > > "index" : "events"  
> > > },  
> > > "state" : "STARTED",  
> > > "index" : {  
> > > "size" : "673.3mb",  
> > > "size\_in\_bytes" : 706043580  
> > > },  
> > > "translog" : {  
> > > "id" : 1361432592857,  
> > > "operations" : 0  
> > > },  
> > > "docs" : {  
> > > "num\_docs" : 1869131,  
> > > "max\_doc" : 2263281,  
> > > "deleted\_docs" : 394150  
> > > },  
> > > "merges" : {  
> > > "current" : 0,  
> > > "current\_docs" : 0,  
> > > "current\_size" : "0b",  
> > > "current\_size\_in\_bytes" : 0,  
> > > "total" : 1,  
> > > "total\_time" : "3.7s",  
> > > "total\_time\_in\_millis" : 3776,  
> > > "total\_docs" : 2263281,  
> > > "total\_size" : "0b",  
> > > "total\_size\_in\_bytes" : 0  
> > > },  
> > > "refresh" : {  
> > > "total" : 1,  
> > > "total\_time" : "0s",  
> > > "total\_time\_in\_millis" : 0  
> > > },  
> > > "flush" : {  
> > > "total" : 1,  
> > > "total\_time" : "18ms",  
> > > "total\_time\_in\_millis" : 18  
> > > }  
> > > } ]  
> > > }  
> > > }  
> > > }
> > > 
> > > \_segments
> > > 
> > > {  
> > > "ok" : true,  
> > > "\_shards" : {  
> > > "total" : 1,  
> > > "successful" : 1,  
> > > "failed" : 0  
> > > },  
> > > "indices" : {  
> > > "events" : {  
> > > "shards" : {  
> > > "0" : [ {  
> > > "routing" : {  
> > > "state" : "STARTED",  
> > > "primary" : true,  
> > > "node" : "vwbSeFjtQ3SqdbJzh8V-uA"  
> > > },  
> > > "num\_committed\_segments" : 20,  
> > > "num\_search\_segments" : 20,  
> > > "segments" : {  
> > > "\_4qm" : {  
> > > "generation" : 6142,  
> > > "num\_docs" : 6730,  
> > > "deleted\_docs" : 67313,  
> > > "size" : "29.8mb",  
> > > "size\_in\_bytes" : 31342776,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_ukk" : {  
> > > "generation" : 39620,  
> > > "num\_docs" : 579895,  
> > > "deleted\_docs" : 232003,  
> > > "size" : "203.9mb",  
> > > "size\_in\_bytes" : 213859775,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1o32" : {  
> > > "generation" : 77870,  
> > > "num\_docs" : 1131181,  
> > > "deleted\_docs" : 71964,  
> > > "size" : "389.9mb",  
> > > "size\_in\_bytes" : 408880404,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1o3e" : {  
> > > "generation" : 77882,  
> > > "num\_docs" : 17179,  
> > > "deleted\_docs" : 15495,  
> > > "size" : "9.9mb",  
> > > "size\_in\_bytes" : 10401244,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1qse" : {  
> > > "generation" : 81374,  
> > > "num\_docs" : 64627,  
> > > "deleted\_docs" : 1870,  
> > > "size" : "16.5mb",  
> > > "size\_in\_bytes" : 17305817,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1qyf" : {  
> > > "generation" : 81591,  
> > > "num\_docs" : 9475,  
> > > "deleted\_docs" : 2073,  
> > > "size" : "3.7mb",  
> > > "size\_in\_bytes" : 3894229,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1r7a" : {  
> > > "generation" : 81910,  
> > > "num\_docs" : 15972,  
> > > "deleted\_docs" : 2375,  
> > > "size" : "5mb",  
> > > "size\_in\_bytes" : 5267040,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1rck" : {  
> > > "generation" : 82100,  
> > > "num\_docs" : 9818,  
> > > "deleted\_docs" : 6,  
> > > "size" : "3.5mb",  
> > > "size\_in\_bytes" : 3753253,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1re0" : {  
> > > "generation" : 82152,  
> > > "num\_docs" : 16651,  
> > > "deleted\_docs" : 787,  
> > > "size" : "4.6mb",  
> > > "size\_in\_bytes" : 4896408,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1rhz" : {  
> > > "generation" : 82295,  
> > > "num\_docs" : 14006,  
> > > "deleted\_docs" : 264,  
> > > "size" : "4.7mb",  
> > > "size\_in\_bytes" : 5016838,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1ri0" : {  
> > > "generation" : 82296,  
> > > "num\_docs" : 479,  
> > > "deleted\_docs" : 0,  
> > > "size" : "162.7kb",  
> > > "size\_in\_bytes" : 166651,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1ri4" : {  
> > > "generation" : 82300,  
> > > "num\_docs" : 266,  
> > > "deleted\_docs" : 0,  
> > > "size" : "110.1kb",  
> > > "size\_in\_bytes" : 112763,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1ri6" : {  
> > > "generation" : 82302,  
> > > "num\_docs" : 250,  
> > > "deleted\_docs" : 0,  
> > > "size" : "109.3kb",  
> > > "size\_in\_bytes" : 111965,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1ri8" : {  
> > > "generation" : 82304,  
> > > "num\_docs" : 153,  
> > > "deleted\_docs" : 0,  
> > > "size" : "85.4kb",  
> > > "size\_in\_bytes" : 87460,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1ria" : {  
> > > "generation" : 82306,  
> > > "num\_docs" : 549,  
> > > "deleted\_docs" : 0,  
> > > "size" : "190.7kb",  
> > > "size\_in\_bytes" : 195340,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1riu" : {  
> > > "generation" : 82326,  
> > > "num\_docs" : 261,  
> > > "deleted\_docs" : 0,  
> > > "size" : "106kb",  
> > > "size\_in\_bytes" : 108604,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1rj6" : {  
> > > "generation" : 82338,  
> > > "num\_docs" : 533,  
> > > "deleted\_docs" : 0,  
> > > "size" : "191.1kb",  
> > > "size\_in\_bytes" : 195748,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1rj8" : {  
> > > "generation" : 82340,  
> > > "num\_docs" : 276,  
> > > "deleted\_docs" : 0,  
> > > "size" : "119.4kb",  
> > > "size\_in\_bytes" : 122299,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1rjb" : {  
> > > "generation" : 82343,  
> > > "num\_docs" : 251,  
> > > "deleted\_docs" : 0,  
> > > "size" : "100.7kb",  
> > > "size\_in\_bytes" : 103129,  
> > > "committed" : true,  
> > > "search" : true  
> > > },  
> > > "\_1rjs" : {  
> > > "generation" : 82360,  
> > > "num\_docs" : 579,  
> > > "deleted\_docs" : 0,  
> > > "size" : "207.2kb",  
> > > "size\_in\_bytes" : 212180,  
> > > "committed" : true,  
> > > "search" : true  
> > > }  
> > > }  
> > > } ]  
> > > }  
> > > }  
> > > }  
> > > }
> > > 
> > > On Monday, February 25, 2013 6:22:31 AM UTC-8, simonw wrote:
> > > 
> > > > Hey Taras,
> > > > 
> > > > it would be awesome if you could provide some more infos about you  
> > > > problems so we can fix it quickly
> > > > 
> > > > simon
> > > > 
> > > > On Monday, February 25, 2013 9:42:20 AM UTC+1, simonw wrote:
> > > > 
> > > > > hey,
> > > > > 
> > > > > can you give some more background here, like did you start this on an  
> > > > > old index ie. an index you created with a previous version of ES / Lucene?  
> > > > > Can you reproduce this?
> > > > > 
> > > > > simon
> > > > > 
> > > > > On Monday, February 25, 2013 7:54:46 AM UTC+1, Taras Shkvarchuk wrote:
> > > > > 
> > > > > > While testing the master build with lucene 4.1, I got a permanent  
> > > > > > merge error filling up my logs.  
> > > > > > Has anyone else seen it? Anything I can provide to help debug?
> > > > > > 
> > > > > > [2013-02-24 22:43:38,863][WARN][index.merge.scheduler]  
> > > > > > [Enchantress] [events][0] failed to merge  
> > > > > > java.lang.ArrayIndexOutOfBoundsException  
> > > > > > at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:132)  
> > > > > > at  
> > > > > > org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:135)  
> > > > > > at  
> > > > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$ChunkIterator.decompress(CompressingStoredFieldsReader.java:388)  
> > > > > > at  
> > > > > > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:368)  
> > > > > > at  
> > > > > > org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:283)  
> > > > > > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:105)  
> > > > > > at  
> > > > > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3698)  
> > > > > > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3303)  
> > > > > > at  
> > > > > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)  
> > > > > > at  
> > > > > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:90)  
> > > > > > at  
> > > > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
> > > > > > 
> > > > > > Thanks!
> > > > > > 
> > > > > > -Taras

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:23am UTC](https://discuss.elastic.co/t/merge-error-with-lucene-4-1/10873/9 "2017-07-06T02:23:57Z")

</div>


