"Format version is not supported"

(Lars Tobias Skjong Børsting) #1


I upgraded from Elasticsearch 2.1.1 to 5.1.1, then by mistake down to 2.1.1 again, and back up to 5.1.1.

Now, two of my indices are out of services, with reason below. What can I do to recover?

$ curl -XGET 'http://localhost:9200/_cluster/allocation/explain?pretty' -d'{  "index": "logstash-2016.12.14",  "shard": 1,  "primary": true}'
  "shard" : {
    "index" : "logstash-2016.12.14",
    "index_uuid" : "78iFiCffRAKSBu7uf4Ekuw",
    "id" : 1,
    "primary" : true
  "assigned" : false,
  "shard_state_fetch_pending" : false,
  "unassigned_info" : {
    "reason" : "CLUSTER_RECOVERED",
    "at" : "2016-12-14T13:37:20.867Z",
    "delayed" : false,
    "allocation_status" : "no_valid_shard_copy"
  "allocation_delay_in_millis" : 60000,
  "remaining_delay_in_millis" : 0,
  "nodes" : {
    "H4UxrgYbSzmcT2UE3WCJnw" : {
      "node_name" : "H4UxrgY",
      "node_attributes" : { },
      "store" : {
        "shard_copy" : "CORRUPT",
        "store_exception" : "CorruptIndexException[failed engine (reason: [corrupt file (source: [start])]) (resource=preexisting_corruption)]; nested: IOException[failed engine (reason: [corrupt file (source: [start])])]; nested: IndexFormatTooNewException[Format version is not supported (resource MMapIndexInput(path=\"/usr/share/elasticsearch/data/elasticsearch/nodes/0/indices/78iFiCffRAKSBu7uf4Ekuw/1/index/_5xh.cfs\") [slice=_5xh.dim]): 3 (needs to be between 0 and 2)]; "
      "final_decision" : "NO",
      "final_explanation" : "the copy of the shard is corrupt",
      "weight" : 36.25,
      "decisions" : [ ]

(David Pilato) #2

Bad luck here. :frowning:

Apart trying again to have all your cluster running 5.0.1 I'm not sure how else you can recover from this situation.
I let other people commenting but worse case I'd delete this index.
Apparently it's about logs logstash-2016.12.14 so may be you can afford it without reindexing?

BTW how many nodes do you have? Is node H4UxrgY a 5.0 one? I guess so judging by its name.

(Lars Tobias Skjong Børsting) #3

Hi, Thanks for your help. Yes, there is only the one node in the cluster. I'm not sure if the index is really corrupted, it just says it has the wrong version (too high). Is there no way to "force" it to be allocated with the newest version?

(David Pilato) #4

Sorry. I don't know.

May @jpountz has an idea?

(Adrien Grand) #5

This version check is used by Lucene to detect when users try to read an index with an older version than the one that was used to create it.

Version 3 of the BKD tree maps to https://issues.apache.org/jira/browse/LUCENE-7501 which was added to Lucene 6.3, and Elasticsearch 5.1.1 is the first version to use this version of Lucene.

Are you sure you are running 5.1.1 and not an older 5.0 release? The reason I am asking is that you would get this error if you were on 5.0, but not on 5.1.

What do GET /_cat/nodes?v&h=id,name,v and GET logstash-2016.12.14/_settings return?

(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.