# Transform date field (timeStamp) "failed to parse field" **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-machine-learning\ **Created:** [April 15, 2020, 7:03pm UTC](https://discuss.elastic.co/t/transform-date-field-timestamp-failed-to-parse-field/228192 "2020-04-15T19:03:03Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Cain310](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cain310/32/53115_2.png) [@Cain310](https://discuss.elastic.co/u/Cain310)\ **Post date:** [April 15, 2020, 7:03pm UTC](https://discuss.elastic.co/t/transform-date-field-timestamp-failed-to-parse-field/228192/1 "2020-04-15T19:03:03Z")
Hi I am having trouble with transforming a timeStamp date field for ml. I am getting this error in the logs failed to parse field [doc.timeStamp.min] of type [date] in document with id 'xxx'. Preview of field's value: '2019-11-28 09:35:05' Ive tried creating a mapping dynamic template and setting this fields format prior to running the transform ```auto "dynamic_templates": [ { "dates": { "match": ".*timeStamp*", "match_pattern": "regex", "mapping": { "format": "yyyy-MM-dd HH:mm:ss||dateOptionalTime||MM/dd/yyyy||yyyy-MM-dd", "ignore_malformed": true, "type": "date" } } } ] ``` As well as attempting to set the format in advanced pivot editor. ```auto "doc.timeStamp.min": { "min": { "field": "doc.timeStamp", "format": "yyyy-MM-dd HH:mm:ss||dateOptionalTime||MM/dd/yyyy||yyyy-MM-dd" } }, ``` stumped.. ---
**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\ **Post date:** [April 16, 2020, 5:22am UTC](https://discuss.elastic.co/t/transform-date-field-timestamp-failed-to-parse-field/228192/2 "2020-04-16T05:22:56Z")
This sounds like a bug to me. For reproduction it would be good to have a full data sample: Can you post an example document, at least an example for `doc.timeStamp`? Which version are you on? Transform deducts mappings from the source and creates a mapping according to that. You can see the deduced mappings in the output of `_preview`. Please run `_preview` in dev console, you can copy the command from the UI using the upper right icon in the preview table. For customizing mappings, create the index yourself, index templates are _not_ taken into account. You can use the output of `_preview` for this. ---
**Author:** ![Cain310](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cain310/32/53115_2.png) [@Cain310](https://discuss.elastic.co/u/Cain310)\ **Post date:** [April 16, 2020, 3:26pm UTC](https://discuss.elastic.co/t/transform-date-field-timestamp-failed-to-parse-field/228192/3 "2020-04-16T15:26:32Z")
Thanks for the help Hendrik, I created the mapping manually before hand using the preview and was able to set the date format manually in dev tools. Probably a bug with the transform ui as the source doc.timeStamp I'm pulling from was the same. Also the transform preview shows the date parsed correctly prior to begining the transform which is strange. I will test my ml jobs hopefully they work the same. For reference i am using 7.6.2 Source doc.timeStamp mapping ```auto "timeStamp" : { "type" : "date", "ignore_malformed" : true, "format" : "yyyy-MM-dd HH:mm:ss||dateOptionalTime||MM/dd/yyyy||yyyy-MM-dd" }, ``` I also noticed the resulting mapping of the transform now looks different in comparision to when i did not set the mapping manually. ```auto { "xxx" : { "mappings" : { "dynamic_templates" : [ { "dates" : { "match" : ".*timeStamp*", "match_pattern" : "regex", "mapping" : { "format" : "yyyy-MM-dd HH:mm:ss||dateOptionalTime||MM/dd/yyyy||yyyy-MM-dd", "ignore_malformed" : true, "type" : "date" } } } ], "properties" : { "xxx" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "xxx" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "xxx" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "xxx" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "xxx" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "xxx" : { "type" : "double" }, "xxx" : { "type" : "long" }, "xxx" : { "type" : "long" }, "doc" : { "properties" : { "xxx" : { "properties" : { "keyword" : { "type" : "keyword" } } }, "timeStamp" : { "properties" : { "max" : { "type" : "date", "ignore_malformed" : true, "format" : "yyyy-MM-dd HH:mm:ss||dateOptionalTime||MM/dd/yyyy||yyyy-MM-dd" }, "min" : { "type" : "date", "ignore_malformed" : true, "format" : "yyyy-MM-dd HH:mm:ss||dateOptionalTime||MM/dd/yyyy||yyyy-MM-dd" } } } } }, "xxx" : { "properties" : { "sum" : { "type" : "double" } } } } } } } ``` Where before it omitted any scripted field mappings. Not sure if this is purposefully done. ```auto { "xxx" : { "mappings" : { "_meta" : { "created_by" : "transform", "_transform" : { "transform" : "xxx", "version" : { "created" : "7.6.2" }, "creation_date_in_millis" : 1586896729706 } }, "properties" : { "xxx" : { "type" : "double" }, "xxx" : { "type" : "long" }, "xxx" : { "type" : "long" }, "doc" : { "properties" : { "xxx" : { "properties" : { "keyword" : { "type" : "keyword" } } }, "timeStamp" : { "properties" : { "max" : { "type" : "date" }, "min" : { "type" : "date" } } } } }, "xxx" : { "properties" : { "sum" : { "type" : "double" } } } } } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 14, 2020, 3:26pm UTC](https://discuss.elastic.co/t/transform-date-field-timestamp-failed-to-parse-field/228192/4 "2020-05-14T15:26:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.