Corrupted indexes?


(Paul Brown) #1

Hi, ElasticSearchFolk --

I've seen some index corruption issues show up in the last few days in a four-node cluster with a good number of indexes and stored documents:

--- SNIP ---
[20:18:01,744][WARN ][index.merge.scheduler ] [Storm, Johnny] [redacted][3] failed to merge
org.apache.lucene.index.CorruptIndexException: docs out of order (1367 <= 1367 )
at org.apache.lucene.index.FormatPostingsDocsWriter.addDoc(FormatPostingsDocsWriter.java:84)
--- SNIP ---

This is with 0.17.9 running on ec2 with the S3 gateway; configured for six shards and one replica. The configuration is otherwise stock.

There is an open Lucene issue and some discussion:

https://issues.apache.org/jira/browse/LUCENE-2501
http://lucene.472066.n3.nabble.com/CorruptIndexException-when-indexing-td2254889.html

that point to RAM buffer settings. Looking at the code in FormatPostingsDocsWriter, it could be the result of integer overflow.

Ring a bell or happen to anyone else?

-- Paul


(system) #2