# Index "failed to merge"
**URL:** https://discuss.elastic.co/t/index-failed-to-merge/5661
**Category:** Elasticsearch
**Created:** [October 22, 2011, 7:39pm UTC](https://discuss.elastic.co/t/index-failed-to-merge/5661 "2011-10-22T19:39:59Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Kenneth\_Loafman](https://discuss.elastic.co/u/Kenneth_Loafman)
#### Post date: [October 22, 2011, 7:39pm UTC](https://discuss.elastic.co/t/index-failed-to-merge/5661/1 "2011-10-22T19:39:59Z")
Hi,
We're got the following problem on the 15th:
[2011-10-15 16:10:19,023][WARN][index.merge.scheduler] [Rand, Daniel]
[co0181ca0607][0] failed to merge
org.apache.lucene.index.CorruptIndexException: docs out of order (221 \<= 221
)
at
org.apache.lucene.index.FormatPostingsDocsWriter.addDoc(FormatPostingsDocsWriter.java:84)
at
org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:590)
at
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:538)
at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:470)
at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4273)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3917)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
at
org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
After that, it's just a one line warning message ever so often.
We have 2 shards and one replica, so the data should be safe, right?
How do I fix this?
...Thanks,
...Ken
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [October 23, 2011, 12:26am UTC](https://discuss.elastic.co/t/index-failed-to-merge/5661/2 "2011-10-23T00:26:22Z")
You can restart the node that causes this problem, and have the shards on it
resync. Though it should not happen, any other failures before it happened?
Can you gist the logs for it from around the time it just started?
On Sat, Oct 22, 2011 at 9:39 PM, Kenneth Loafman
[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)wrote:
> Hi,
>
> We're got the following problem on the 15th:
> [2011-10-15 16:10:19,023][WARN][index.merge.scheduler] [Rand, Daniel]
> [co0181ca0607][0] failed to merge
> org.apache.lucene.index.CorruptIndexException: docs out of order (221 \<=
> 221 )
> at
> org.apache.lucene.index.FormatPostingsDocsWriter.addDoc(FormatPostingsDocsWriter.java:84)
> at
> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:590)
> at
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:538)
> at
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:470)
> at
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4273)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3917)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
> at
> org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
>
> After that, it's just a one line warning message ever so often.
>
> We have 2 shards and one replica, so the data should be safe, right?
>
> How do I fix this?
>
> ...Thanks,
> ...Ken
---
### Author:  [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)
#### Post date: [October 23, 2011, 12:41pm UTC](https://discuss.elastic.co/t/index-failed-to-merge/5661/3 "2011-10-23T12:41:59Z")
Looks like a restart caused the original problem:
>
I tried the restart node and no go:
>
On Sat, Oct 22, 2011 at 7:26 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> You can restart the node that causes this problem, and have the shards on
> it resync. Though it should not happen, any other failures before it
> happened? Can you gist the logs for it from around the time it just started?
>
> On Sat, Oct 22, 2011 at 9:39 PM, Kenneth Loafman \<
> [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
>
> > Hi,
> >
> > We're got the following problem on the 15th:
> > [2011-10-15 16:10:19,023][WARN][index.merge.scheduler] [Rand, Daniel]
> > [co0181ca0607][0] failed to merge
> > org.apache.lucene.index.CorruptIndexException: docs out of order (221 \<=
> > 221 )
> > at
> > org.apache.lucene.index.FormatPostingsDocsWriter.addDoc(FormatPostingsDocsWriter.java:84)
> > at
> > org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:590)
> > at
> > org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:538)
> > at
> > org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:470)
> > at
> > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> > at
> > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4273)
> > at
> > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3917)
> > at
> > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
> > at
> > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
> > at
> > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
> >
> > After that, it's just a one line warning message ever so often.
> >
> > We have 2 shards and one replica, so the data should be safe, right?
> >
> > How do I fix this?
> >
> > ...Thanks,
> > ...Ken
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [October 23, 2011, 8:17pm UTC](https://discuss.elastic.co/t/index-failed-to-merge/5661/4 "2011-10-23T20:17:19Z")
I will look at the initial failure that might have caused it, but for now,
can you stop the node, delete the co0181ca0607][0] shard location (under
data/nodes/0/indices/co0181ca0607/0, and then start it back up? It will
force the shard to be copied from the other node.
On Sun, Oct 23, 2011 at 2:41 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> Looks like a restart caused the original problem:
> [Index "failed to merge" · GitHub](https://gist.github.com/1307321#file_original_fail.txt)
>
> I tried the restart node and no go:
> [Index "failed to merge" · GitHub](https://gist.github.com/1307321#file_restart_fail.txt)
>
> On Sat, Oct 22, 2011 at 7:26 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
>
> > You can restart the node that causes this problem, and have the shards on
> > it resync. Though it should not happen, any other failures before it
> > happened? Can you gist the logs for it from around the time it just started?
> >
> > On Sat, Oct 22, 2011 at 9:39 PM, Kenneth Loafman \<
> > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> >
> > > Hi,
> > >
> > > We're got the following problem on the 15th:
> > > [2011-10-15 16:10:19,023][WARN][index.merge.scheduler] [Rand,
> > > Daniel] [co0181ca0607][0] failed to merge
> > > org.apache.lucene.index.CorruptIndexException: docs out of order (221 \<=
> > > 221 )
> > > at
> > > org.apache.lucene.index.FormatPostingsDocsWriter.addDoc(FormatPostingsDocsWriter.java:84)
> > > at
> > > org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:590)
> > > at
> > > org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:538)
> > > at
> > > org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:470)
> > > at
> > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> > > at
> > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4273)
> > > at
> > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3917)
> > > at
> > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
> > > at
> > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
> > > at
> > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
> > >
> > > After that, it's just a one line warning message ever so often.
> > >
> > > We have 2 shards and one replica, so the data should be safe, right?
> > >
> > > How do I fix this?
> > >
> > > ...Thanks,
> > > ...Ken
---
### Author:  [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)
#### Post date: [October 24, 2011, 11:30am UTC](https://discuss.elastic.co/t/index-failed-to-merge/5661/5 "2011-10-24T11:30:33Z")
Thanks! That worked.
On Sun, Oct 23, 2011 at 3:17 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> I will look at the initial failure that might have caused it, but for now,
> can you stop the node, delete the co0181ca0607][0] shard location (under
> data/nodes/0/indices/co0181ca0607/0, and then start it back up? It will
> force the shard to be copied from the other node.
>
> On Sun, Oct 23, 2011 at 2:41 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
>
> > Looks like a restart caused the original problem:
> > [Index "failed to merge" · GitHub](https://gist.github.com/1307321#file_original_fail.txt)
> >
> > I tried the restart node and no go:
> > [Index "failed to merge" · GitHub](https://gist.github.com/1307321#file_restart_fail.txt)
> >
> > On Sat, Oct 22, 2011 at 7:26 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> >
> > > You can restart the node that causes this problem, and have the shards on
> > > it resync. Though it should not happen, any other failures before it
> > > happened? Can you gist the logs for it from around the time it just started?
> > >
> > > On Sat, Oct 22, 2011 at 9:39 PM, Kenneth Loafman \<
> > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > >
> > > > Hi,
> > > >
> > > > We're got the following problem on the 15th:
> > > > [2011-10-15 16:10:19,023][WARN][index.merge.scheduler] [Rand,
> > > > Daniel] [co0181ca0607][0] failed to merge
> > > > org.apache.lucene.index.CorruptIndexException: docs out of order (221 \<=
> > > > 221 )
> > > > at
> > > > org.apache.lucene.index.FormatPostingsDocsWriter.addDoc(FormatPostingsDocsWriter.java:84)
> > > > at
> > > > org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:590)
> > > > at
> > > > org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:538)
> > > > at
> > > > org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:470)
> > > > at
> > > > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
> > > > at
> > > > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4273)
> > > > at
> > > > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3917)
> > > > at
> > > > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
> > > > at
> > > > org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:88)
> > > > at
> > > > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
> > > >
> > > > After that, it's just a one line warning message ever so often.
> > > >
> > > > We have 2 shards and one replica, so the data should be safe, right?
> > > >
> > > > How do I fix this?
> > > >
> > > > ...Thanks,
> > > > ...Ken
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:51am UTC](https://discuss.elastic.co/t/index-failed-to-merge/5661/6 "2017-07-06T03:51:05Z")