Corrupted indexes?

(Paul Brown) #1

Hi, ElasticSearchFolk --

I've seen some index corruption issues show up in the last few days in a four-node cluster with a good number of indexes and stored documents:

--- SNIP ---
[20:18:01,744][WARN ][index.merge.scheduler ] [Storm, Johnny] [redacted][3] failed to merge
org.apache.lucene.index.CorruptIndexException: docs out of order (1367 <= 1367 )
at org.apache.lucene.index.FormatPostingsDocsWriter.addDoc(
--- SNIP ---

This is with 0.17.9 running on ec2 with the S3 gateway; configured for six shards and one replica. The configuration is otherwise stock.

There is an open Lucene issue and some discussion:

that point to RAM buffer settings. Looking at the code in FormatPostingsDocsWriter, it could be the result of integer overflow.

Ring a bell or happen to anyone else?

-- Paul

(system) #2