Reindex causes mapper_parsing_exception

[Gavin_Hardy]

Hello,

We are currently using Functionbeat to create our templates, and yesterday we started seeing errors. Looking at the index's that were affected, it appears someone had create a custom template, which caused the errors. So I put the index back to using the original template, and now when I go to reindex the old index, to use the updated template, it is complaining about the following error.

{
 "source": {
 "index": "is-prod-fb-2020.01.29-000235"
  },
  "dest": {
  "index": "is-prod-fb-2020.01.29-000235-tmp"
  }
}

    {
      "index": "is-fb-2020.01.29-000235-tmp",
      "type": "_doc",
      "id": "cTiX8W8BNtz7_4jyhWdW",
      "cause": {
        "type": "mapper_parsing_exception",
        "reason": "object mapping for [error] tried to parse field [error] as object, but found a concrete value"
      },
      "status": 400
    }
  ]

I understand the data inside the index, is of type string, and not object. Which is why it wont index, but my question is, how can I make it index, or change the data to be Object, without losing the index?

Any help is much appreciated!

[Gavin_Hardy]

Just to add a note, all new shards use the updated template are working as expected, its just 4 shards that are still causing issues due to using the old template.

[spinscale (Alexander Reelsen)]

See quick solution is to use ignore_malformed in the mapping so a document gets indexed. However if you are reindexing into a new index, what prevents you from fixing the mapping in that index. I might be missing something there... also if the initial structure is wrong, you can use a script to fix the document structure and move that string based error field somewhere else and then you would end up with clean, searchable data.

[Gavin_Hardy]

So, when I try to fix the mapping in those index, I get this error

{
  "properties": {
    "error": {
      "type": "object"
    }
  }
}

Which throws this error

  "error": {
    "root_cause": [
      {
        "type": "remote_transport_exception",
        "reason": "[instance-0000000021][172.17.0.32:19849][indices:admin/mapping/put]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "mapper [error] of different type, current_type [text], merged_type [ObjectMapper]"
  },
  "status": 400
}

[spinscale (Alexander Reelsen)]

you cannot change the mapping field type of an existing index, you need to reindex.

[Gavin_Hardy]

Then I just get the error above when trying to reindex. Lost at what to do

[spinscale (Alexander Reelsen)]

have you created the destination index with your new mapping first and verified that the mapping is what you expect it to be?

[Gavin_Hardy]

Hey, how do I create a destination index with the new mapping? When I updated the index to use the new template, I forced the index to roll_over, and confirmed all new shards worked as expected, I just have these 4 shards, that are still using the old template, that are not working.

POST /is-prod-fb/_rollover 
{
  "conditions": {
    "max_age":   "1d",
    "max_size":  "1gb"
  }
}