Unable to reindex after field changes

At the end of each month to keep index and shard counts low, we reindex all of our daily indices into a larger monthly index. We do this by manually running a reindex job on each of our larger index groups by running the following command in DevTools in Kibana:

    POST _reindex
{
  "source": {
    "index": ["daily-index_2020-07-06",
    "daily-index_2020-07-23",
    "daily-index_2020-07-27",
    "daily-index_2020-07-31"
]
  },
  "dest": {
    "index": "monthly-index_2020-07-reindex"
  }
}

Recently, the devs had to make some changes to certain fields for analytics reasons. Now upon reindexing I'm given the following error:

"cause" : {
        "type" : "mapper_parsing_exception",
        "reason" : "object mapping for [responseVisual] tried to parse field [responseVisual] as object, but found a concrete value"
      },
      "status" : 400

How can I either fix this so I can reindex again or ignore these errors(assuming they still show up in Kibana correctly).

We're looking into Curator and Index Lifecycle policies to automate this in the future, but we aren't there yet.

Thank you.

@cjcenizal thanks :slight_smile:

Hi Gavin, this sounds similar to this StackOverflow question: https://stackoverflow.com/questions/30454108/how-to-solve-mapperparsingexception-object-mapping-for-test-tried-to-parse-as

The error is saying that one of your indices has responseVisual mapped as an object, and another has it mapped as a non-object value. These are incompatible mapping types. I suggest you either only reindex indices that have compatible mappings, or reindex using an ingest node pipeline to normalize your data in different indices to the same type. There is a section in the reindex docs that mentions this. You can learn more about ingest node pipelines here. Does this help?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.