# ConcurrentMergeScheduler exceptions in the log

**URL:** <https://discuss.elastic.co/t/concurrentmergescheduler-exceptions-in-the-log/3830>\
**Category:** Elasticsearch\
**Created:** [January 26, 2011, 7:20am UTC](https://discuss.elastic.co/t/concurrentmergescheduler-exceptions-in-the-log/3830 "2011-01-26T07:20:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vpunski](https://avatars.discourse-cdn.com/v4/letter/v/54ee81/32.png) [@vpunski](https://discuss.elastic.co/u/vpunski)\
**Post date:** [January 26, 2011, 7:20am UTC](https://discuss.elastic.co/t/concurrentmergescheduler-exceptions-in-the-log/3830/1 "2011-01-26T07:20:58Z")

</div>

Hi,  
we encounter a lot of exceptions of two types (see below) in our log  
files, together with huge performance degradation for index operations  
of our 6-node cluster.  
The status remains "green", the system continues to function.  
Any suggestions will be appreciated.  
Please let me know if you need more information.

Regards,  
Vadim

* * *

org.apache.lucene.index.MergePolicy$MergeException:  
java.io.IOException: Illegal seek  
at  
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:  
347)  
at org.apache.lucene.index.ConcurrentMergeScheduler  
$MergeThread.run(ConcurrentMergeScheduler.java:312)  
Caused by: java.io.IOException: Illegal seek  
at sun.nio.ch.FileDispatcher.pread0(Native Method)  
at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:31)  
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:234)  
at sun.nio.ch.IOUtil.read(IOUtil.java:210)  
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:622)  
at org.apache.lucene.store.NIOFSDirectory  
$NIOFSIndexInput.readInternal(NIOFSDirectory.java:168)  
at  
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:  
160)  
at  
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:  
119)  
at  
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:  
94)  
at  
org.apache.lucene.store.IndexOutput.copyBytes(IndexOutput.java:174)  
at  
org.apache.lucene.index.FieldsWriter.addRawDocuments(FieldsWriter.java:  
216)  
at  
org.apache.lucene.index.SegmentMerger.copyFieldsNoDeletions(SegmentMerger.java:  
430)  
at  
org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:  
342)  
at  
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:155)  
at  
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4443)  
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:  
4000)  
at  
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:  
231)  
at org.apache.lucene.index.ConcurrentMergeScheduler  
$MergeThread.run(ConcurrentMergeScheduler.java:288)

* * *

org.apache.lucene.index.MergePolicy$MergeException:  
java.lang.AssertionError: Terms are out of order: field=obj.key  
(number 54) lastField=obj.key (number 54)  
text=lba0000020031229dzct001V1 lastText=lba0000020031229dzct001pm  
at  
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:  
347)  
at org.apache.lucene.index.ConcurrentMergeScheduler  
$MergeThread.run(ConcurrentMergeScheduler.java:312)  
Caused by: java.lang.AssertionError: Terms are out of order:  
field=obj.key (number 54) lastField=obj.key (number 54)  
text=lba0000020031229dzct001V1 lastText=lba0000020031229dzct001pm  
at  
org.apache.lucene.index.TermInfosWriter.add(TermInfosWriter.java:163)  
at  
org.apache.lucene.index.FormatPostingsDocsWriter.finish(FormatPostingsDocsWriter.java:  
115)  
at  
org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:  
707)  
at  
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:  
635)  
at  
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:  
573)  
at  
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:156)  
at  
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4443)  
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:  
4000)  
at  
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:  
231)  
at org.apache.lucene.index.ConcurrentMergeScheduler  
$MergeThread.run(ConcurrentMergeScheduler.java:288)

* * *

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [January 26, 2011, 9:59am UTC](https://discuss.elastic.co/t/concurrentmergescheduler-exceptions-in-the-log/3830/2 "2011-01-26T09:59:49Z")

</div>

Which version of ES are you using?  
On Wednesday, January 26, 2011 at 9:20 AM, vadim wrote:

> Hi,  
> we encounter a lot of exceptions of two types (see below) in our log  
> files, together with huge performance degradation for index operations  
> of our 6-node cluster.  
> The status remains "green", the system continues to function.  
> Any suggestions will be appreciated.  
> Please let me know if you need more information.
> 
> Regards,  
> Vadim
> 
> * * *
> 
> org.apache.lucene.index.MergePolicy$MergeException:  
> java.io.IOException: Illegal seek  
> at  
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:  
> 347)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:312)  
> Caused by: java.io.IOException: Illegal seek  
> at sun.nio.ch.FileDispatcher.pread0(Native Method)  
> at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:31)  
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:234)  
> at sun.nio.ch.IOUtil.read(IOUtil.java:210)  
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:622)  
> at org.apache.lucene.store.NIOFSDirectory  
> $NIOFSIndexInput.readInternal(NIOFSDirectory.java:168)  
> at  
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:  
> 160)  
> at  
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:  
> 119)  
> at  
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:  
> 94)  
> at  
> org.apache.lucene.store.IndexOutput.copyBytes(IndexOutput.java:174)  
> at  
> org.apache.lucene.index.FieldsWriter.addRawDocuments(FieldsWriter.java:  
> 216)  
> at  
> org.apache.lucene.index.SegmentMerger.copyFieldsNoDeletions(SegmentMerger.java:  
> 430)  
> at  
> org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:  
> 342)  
> at  
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:155)  
> at  
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4443)  
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:  
> 4000)  
> at  
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:  
> 231)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:288)
> 
> * * *
> 
> org.apache.lucene.index.MergePolicy$MergeException:  
> java.lang.AssertionError: Terms are out of order: field=obj.key  
> (number 54) lastField=obj.key (number 54)  
> text=lba0000020031229dzct001V1 lastText=lba0000020031229dzct001pm  
> at  
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:  
> 347)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:312)  
> Caused by: java.lang.AssertionError: Terms are out of order:  
> field=obj.key (number 54) lastField=obj.key (number 54)  
> text=lba0000020031229dzct001V1 lastText=lba0000020031229dzct001pm  
> at  
> org.apache.lucene.index.TermInfosWriter.add(TermInfosWriter.java:163)  
> at  
> org.apache.lucene.index.FormatPostingsDocsWriter.finish(FormatPostingsDocsWriter.java:  
> 115)  
> at  
> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:  
> 707)  
> at  
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:  
> 635)  
> at  
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:  
> 573)  
> at  
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:156)  
> at  
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4443)  
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:  
> 4000)  
> at  
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:  
> 231)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:288)
> 
> * * *

---

<div class="post-metadata">

**Author:** ![vpunski](https://avatars.discourse-cdn.com/v4/letter/v/54ee81/32.png) [@vpunski](https://discuss.elastic.co/u/vpunski)\
**Post date:** [January 26, 2011, 2:27pm UTC](https://discuss.elastic.co/t/concurrentmergescheduler-exceptions-in-the-log/3830/3 "2011-01-26T14:27:40Z")

</div>

0.14.1

On Wed, Jan 26, 2011 at 11:59 AM, Shay Banon  
[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:

> Which version of ES are you using?
> 
> On Wednesday, January 26, 2011 at 9:20 AM, vadim wrote:
> 
> Hi,  
> we encounter a lot of exceptions of two types (see below) in our log  
> files, together with huge performance degradation for index operations  
> of our 6-node cluster.  
> The status remains "green", the system continues to function.  
> Any suggestions will be appreciated.  
> Please let me know if you need more information.
> 
> Regards,  
> Vadim
> 
> * * *
> 
> org.apache.lucene.index.MergePolicy$MergeException:  
> java.io.IOException: Illegal seek  
> at
> 
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:  
> 347)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:312)  
> Caused by: java.io.IOException: Illegal seek  
> at sun.nio.ch.FileDispatcher.pread0(Native Method)  
> at sun.nio.ch.FileDispatcher.pread(FileDispatcher.java:31)  
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:234)  
> at sun.nio.ch.IOUtil.read(IOUtil.java:210)  
> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:622)  
> at org.apache.lucene.store.NIOFSDirectory  
> $NIOFSIndexInput.readInternal(NIOFSDirectory.java:168)  
> at  
> org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:  
> 160)  
> at
> 
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:  
> 119)  
> at
> 
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:  
> 94)  
> at  
> org.apache.lucene.store.IndexOutput.copyBytes(IndexOutput.javaa:174)
> 
> at  
> org.apache.lucene.index.FieldsWriter.addRawDocuments(FieldsWriter.java:  
> 216)  
> at
> 
> org.apache.lucene.index.SegmentMerger.copyFieldsNoDeletions(SegmentMerger.java:  
> 430)  
> at  
> org.apache.lucene.index.SegmentMerger.mergeFFields(SegmentMerger.java:
> 
> 1. 
> 
> at  
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:155)  
> at  
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4443)  
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:  
> 4000)  
> at
> 
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:  
> 231)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:288)
> 
> * * *
> 
> org.apache.lucene.index.MergePolicy$MergeException:  
> java.lang.AssertionError: Terms are out of order: field=obj.key  
> (number 54) lastField=obj.key (number 54)  
> text=lba0000020031229dzct001V1 lastText=lba0000020031229dzct001pm  
> at
> 
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:  
> 347)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:312)  
> Caused by: java.lang.AssertionError: Terms are out of order:  
> field=obj.key (number 54) lastField=obj.key (number 54)  
> text=lba0000020031229dzct001V1 lastText=lba0000020031229dzct001pm  
> at  
> org.apache.lucene.index.TermInfosWriter.add(TermInfosWriter.java:163)  
> at
> 
> org.apache.lucene.index.FormatPostingsDocsWriter.finish(FormatPostingsDocsWriter.java:  
> 115)  
> at  
> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:  
> 707)  
> at  
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:  
> 635)  
> at  
> org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMergerr.java:
> 
> 1. 
> 
> at  
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:156)  
> at  
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4443)  
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:  
> 4000)  
> at
> 
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:  
> 231)  
> at org.apache.lucene.index.ConcurrentMergeScheduler  
> $MergeThread.run(ConcurrentMergeScheduler.java:288)
> 
> * * *

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:13am UTC](https://discuss.elastic.co/t/concurrentmergescheduler-exceptions-in-the-log/3830/4 "2017-07-06T04:13:13Z")

</div>