Reindexing with date format conversion applied on dynamic fields matching regex

Hello Guys.
I have an index A that has fields dt_0 to dt_n (n<=100) that unfortunately are stored as dynamic mappings and represent the Long value of a date. The index B has the same mappings as A but it also has the right mapping for the dt_n fields as shown below.

"dt_1": {
            "type": "date",
            "format": "strict_date_time"
          }

Do you see any way I could use the reindex API to reindex from A to B and convert to date time only those fields that are name dt_*?
Here are the failures I get when I try to perform a basic reindex operation.

{
  "took": 17151,
  "timed_out": false,
  "total": 459696,
  "updated": 104999,
  "created": 18000,
  "deleted": 0,
  "batches": 123,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1,
  "throttled_until_millis": 0,
  "failures": [
    {
      "index": "lc-uat-de1_accessgrant_project_v2",
      "type": "accessible",
      "id": "5eea03cec4839733aa11da6e",
      "cause": {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse [dt_1]",
        "caused_by": {
          "type": "illegal_argument_exception",
          "reason": "Invalid format: \"1579478400000\" is malformed at \"478400000\""
        }
      },
      "status": 400
    }
  ]
}

Thank you!

Hi @Andrei_Gog
You can try using script in reindex. Assuming all values in dt_1 in index A are epoch millis.

{
  "source": {
    "index": "A"
  },
  "dest": {
    "index": "B"
  },
  "script": {
    "source": "long milliSinceEpoch = ctx._source.remove(\"dt_1\"); Instant instant = Instant.ofEpochMilli(milliSinceEpoch); ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, ZoneId.of('Z')); ctx._source.dt_1 = zdt;"
  }
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.