# Date format changes and causes an error in a transform

**URL:** <https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643>\
**Category:** Elasticsearch\
**Tags:** transforms\
**Created:** [March 2, 2022, 3:43pm UTC](https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643 "2022-03-02T15:43:02Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jker](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@Jker](https://discuss.elastic.co/u/Jker)\
**Post date:** [March 2, 2022, 3:43pm UTC](https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643/1 "2022-03-02T15:43:02Z")

</div>

**Hello!**

I'm trying to create a transform from two different indices but keep on running to this error and have tried many different solutions but can't get it to work.

This is the error:

> Failed to index documents into destination index due to permanent error  
> [BulkIndexingException[Bulk index experienced [160] failures and at least 1 irrecoverable [TransformException[Destination index mappings are incompatible with the transform configuration.]; nested: MapperParsingException[failed to parse field [user\_created] of type [date] in document with id '\<doc\_id\>'. Preview of field's value: '1.513493852792E12']; nested: IllegalArgumentException[failed to parse date field [1.513493852792E12] with format [strict\_date\_optional\_time||epoch\_millis]]; nested: DateTimeParseException[Failed to parse with all enclosed parsers];

This is the transform I'm running:

```auto
{
  "source": {
    "index": [
      "<first_index>",
      "<second_index>"
    ]
  },
  "dest": {
    "index": "<combined_index>"
  },
  "pivot": {
    "group_by": {
      "user_id": {
        "terms": {
          "field": "user_id"
        }
      }
    },
    "aggregations": {
      "user_created": {
        "max": {
          "field": "user_created"
        }
      },
      "last_access": {
        "max": {
          "field": "last_access"
        }
      }
    }
  }
}

```

And this is an example of the data I get running the preview:

```auto
    {
      "last_access" : 1.513493852792E12,
      "user_id" : "<user_id_string>",
      "user_created" : "2015-01-24T12:33:18.263Z"
    },

```

In some cases the `user_created` date is also the same format as the `last_access` in this example.

* * *

And this is the mapping in the destination index:

```auto
{
  "properties": {
    "user_id": {
      "type": "keyword"
    },
    "user_created": {
      "type": "date"
    },
    "last_access": {
      "type": "date"
    }
  }
}

```

In both of the indices the dates are completely normal `"yyyy-MM-dd'T'HH:mm:ss.SSSZ"` format but something happens in the transform that changes it to this `1.513493852792E12` kind of format that causes the error. Kind of still figuring out transforms and elastic as a whole.

Is there something that I'm missing or what could cause this kind of behavior? Thanks a lot!

---

<div class="post-metadata">

**Author:** ![sophie\_chang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sophie_chang/32/18008_2.png) [@sophie\_chang](https://discuss.elastic.co/u/sophie_chang)\
**Post date:** [March 2, 2022, 5:21pm UTC](https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643/2 "2022-03-02T17:21:27Z")

</div>

Hi

What version of elastic is this?

Transforms will deduce the mappings of the destination index, based on the mappings from the source indices. If the source index mappings are not the same, it could cause problems. Date formats can be tricky, but lets establish source index mappings first.

Please run

```auto
GET index1,index2/_mapping/field/user_created,last_access

```

Thanks

---

<div class="post-metadata">

**Author:** ![Jker](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@Jker](https://discuss.elastic.co/u/Jker)\
**Post date:** [March 3, 2022, 7:16am UTC](https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643/3 "2022-03-03T07:16:42Z")

</div>

Hey,

It's the latest version 8.0.

The output after running:

```auto
GET index1,index2/_mapping/field/user_created,last_access

```

Output:

```auto
{
  "<index1>" : {
    "mappings" : {
      "last_access" : {
        "full_name" : "last_access",
        "mapping" : {
          "last_access" : {
            "type" : "date"
          }
        }
      }
    }
  },
  "<index2>" : {
    "mappings" : {
      "user_created" : {
        "full_name" : "user_created",
        "mapping" : {
          "user_created" : {
            "type" : "date"
          }
        }
      }
    }
  }
}

```

Thanks!

---

<div class="post-metadata">

**Author:** ![przemekwitek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/przemekwitek/32/79526_2.png) [@przemekwitek](https://discuss.elastic.co/u/przemekwitek)\
**Post date:** [March 3, 2022, 10:15am UTC](https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643/4 "2022-03-03T10:15:35Z")

</div>

Hi,  
Please create **both** source indices with the same mappings (even if no document in `<index1>` contain `user_created` field and vice versa):

```auto
PUT <index1>
{
  "mappings": {
    "properties": {
      "user_id": {
        "type": "keyword"
      },
      "user_created": {
        "type": "date"
      },
      "last_access": {
        "type": "date"
      }
    }
  }
}

PUT <index2>
{
  "mappings": {
    "properties": {
      "user_id": {
        "type": "keyword"
      },
      "user_created": {
        "type": "date"
      },
      "last_access": {
        "type": "date"
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Jker](https://avatars.discourse-cdn.com/v4/letter/j/ccd318/32.png) [@Jker](https://discuss.elastic.co/u/Jker)\
**Post date:** [March 3, 2022, 10:49am UTC](https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643/5 "2022-03-03T10:49:07Z")

</div>

**Hello!**

> Please create both source indices with the same mappings (even if no document in contain user\_created field and vice versa):

Oh my, this worked like a charm!

Thank you very very much for the both of you!

You saved my day, maybe even the whole week. 😃

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 31, 2022, 10:49am UTC](https://discuss.elastic.co/t/date-format-changes-and-causes-an-error-in-a-transform/298643/6 "2022-03-31T10:49:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.