Is it possible to restore a snapshot (s3) and change index mappings?

Elastic Stack Elasticsearch
1

I'm currently trying to upgrade our systems to 2.x from 1.x.

Unfortunately our 1.x fields had dots ('.') in them. I've written a program that replaces the dots for a legal character.

My plan was then to snapshot the dotless data and restore to an es 2.x, but that doesn't work. Even though I've modified the data, the snapshot takes the index mappings with it:

{
  "error" : {
    "root_cause" : [ {
      "type" : "snapshot_restore_exception",
      "reason" : "[s3incremental:201608011118] cannot restore index [status] because it cannot be upgraded"
    } ],
    "type" : "snapshot_restore_exception",
    "reason" : "[s3incremental:201608011118] cannot restore index [status] because it cannot be upgraded",
    "caused_by" : {
      "type" : "illegal_state_exception",
      "reason" : "unable to upgrade the mappings for the index [status], reason: [Field name [error.cause] cannot contain '.']",
      "caused_by" : {
        "type" : "mapper_parsing_exception",
        "reason" : "Field name [error.cause] cannot contain '.'"
      }
    }
  },
  "status" : 500
}

Is there any way of doing a snapshot or restore either without the index mappings, or by somehow mutating the index mappings on the way in so that they don't reference the illegal field names?

2

No, sorry there isn't. 5.0 should support dots in field names again but you won't be able to restore 1.0 indicies into it because 5.0 isn't compatible with the metadata and on disk format.

At this point I think the only way around this is to copy from 1.x to 2.x (or 5.0, depending on your timetable and how quickly 5.0 become ready, which is still not known). That might mean resting into 1.x and then copying if you don't have the data restored anywhere.

For what it is worth 5.0 has reindex-from-remote already in the alphas. And that supports mutating the documents with a script.

3

Thanks!

I'm going to make moving the documents to a new index part of the data migration process. If I delete the old indexes afterwards, then I'm hoping the snapshots will then only have valid mappings in and will work. Fingers crossed!