# Date parsing error in Elasticsearch 7.1 **URL:** \ **Category:** Elasticsearch\ **Created:** [July 9, 2020, 9:21pm UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607 "2020-07-09T21:21:18Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![sharadha\_s](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharadha_s/32/61664_2.png) [@sharadha\_s](https://discuss.elastic.co/u/sharadha_s)\ **Post date:** [July 9, 2020, 9:21pm UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/1 "2020-07-09T21:21:18Z")
We are using the AWS Elasticsearch 7.1 and we see some date formats are not being supported. It used to work in 5.5 and its a issue with only 7.1. **Mapping :** ```auto "dynamic": "strict", "dynamic_templates": [], "properties": { "_all": { "type": "text" }, "age": { "type": "long" }, "endDateTime": { "type": "date", "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'ZZ'||yyyy-MM-dd'T'HH:mm:ss.SSS||yyyy-MM-dd'T'HH:mm:ss.SSSXXX" }, "id": { "type": "keyword", "copy_to": [ "_all" ], "ignore_above": 256 }, "name": { "type": "keyword", "fields": { "default": { "type": "text", "analyzer": "standard" }, "language": { "type": "text", "analyzer": "english" } }, "copy_to": [ "_all" ], "ignore_above": 256 } } } } ``` **Query:** ```auto "from": 0, "size": 10, "query": { "bool": { "must": [ { "constant_score": { "filter": { "range": { "endDateTime": { "from": "2020-07-08T23:48:21.530-07:00", "to": null, "include_lower": false, "include_upper": true, "boost": 1 } } }, "boost": 1 } } ], "adjust_pure_negative": true, "boost": 1 } } } ``` The error is : "root\_cause": [ { "type": "parse\_exception", "reason": "failed to parse date field [2020-07-08T23:48:21.530-07:00] with format [yyyy-MM-dd'T'HH:mm:ss.SSS'ZZ'||yyyy-MM-dd'T'HH:mm:ss.SSSXXX]: [Text '2020-07-08T23:48:21.530-07:00' could not be parsed, unparsed text found at index 23]" } ], "type": "search\_phase\_execution\_exception", "reason": "all shards failed", "phase": "query", "grouped": true, Can anyone please help here? ---
**Author:** ![Vinayak\_Sapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinayak_sapre/32/45939_2.png) [@Vinayak\_Sapre](https://discuss.elastic.co/u/Vinayak_Sapre)\ **Post date:** [July 10, 2020, 6:25am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/2 "2020-07-10T06:25:04Z")
@sharadha_s I don't know the exact reason. But if you change order of patterns the query works. `yyyy-MM-dd'T'HH:mm:ss.SSSXXX||yyyy-MM-dd'T'HH:mm:ss.SSS'ZZ'||yyyy-MM-dd'T'HH:mm:ss.SSS` Even more surprising, if you set "include\_lower" to true, your current format works too. ---
**Author:** ![sharadha\_s](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharadha_s/32/61664_2.png) [@sharadha\_s](https://discuss.elastic.co/u/sharadha_s)\ **Post date:** [July 10, 2020, 7:51am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/3 "2020-07-10T07:51:18Z")
> [@Vinayak\_Sapre](#): > > include\_lower Yes ordering works . But what if we want to add some more custom date formats to the front and we want to search against them? This doesn't seem to work for all the date formats. ---
**Author:** ![Vinayak\_Sapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinayak_sapre/32/45939_2.png) [@Vinayak\_Sapre](https://discuss.elastic.co/u/Vinayak_Sapre)\ **Post date:** [July 10, 2020, 8:32am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/4 "2020-07-10T08:32:57Z")
I am guessing here. It may have something to do with `yyyy-MM-dd'T'HH:mm:ss.SSS` pattern being subset of `yyyy-MM-dd'T'HH:mm:ss.SSSXXX`. Put more restrictive pattern first. Also you should regression for all types of dates when adding a new pattern. ---
**Author:** ![whatgeorgemade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whatgeorgemade/32/103246_2.png) [@whatgeorgemade](https://discuss.elastic.co/u/whatgeorgemade)\ **Post date:** [July 10, 2020, 9:01am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/5 "2020-07-10T09:01:45Z")
I can't test at the moment but I am pretty certain it's the offset (`-07:00`) that's causing the problem. I think `range` queries expect the time in UTC instead of a localised format. Try removing the offset from the timestamp and using the `time_zone` attribute of the `range` query. There's an [example here](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html#range-query-time-zone). Dates are awkward! ---
**Author:** ![sharadha\_s](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharadha_s/32/61664_2.png) [@sharadha\_s](https://discuss.elastic.co/u/sharadha_s)\ **Post date:** [July 10, 2020, 9:08am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/6 "2020-07-10T09:08:22Z")
We use both range and term queries for the dates depending on the requirement. So does it work for both? ---
**Author:** ![sharadha\_s](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharadha_s/32/61664_2.png) [@sharadha\_s](https://discuss.elastic.co/u/sharadha_s)\ **Post date:** [July 10, 2020, 10:15am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/7 "2020-07-10T10:15:39Z")
make sense. But the issue is if we want to search with multiple date formats, how we define this order. Lets i say i have a format defined in this order : ["yyyy-MM-dd'T'HH:mm:ss.SSSXXX","yyyy-MM-dd'T'HH:mm:ss.S", "yyyy-MM-dd'T'HH:mm:ss.SS"]. I want to search against "2020-07-08T23:48:21.530-07:00" or search against "2020-02-04T15:16:03.01". In this case first one will pass as it will match the first format and second one fails which will match the second format. ---
**Author:** ![Vinayak\_Sapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinayak_sapre/32/45939_2.png) [@Vinayak\_Sapre](https://discuss.elastic.co/u/Vinayak_Sapre)\ **Post date:** [July 10, 2020, 2:41pm UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/8 "2020-07-10T14:41:47Z")
@sharadha_s swapping patterns 2 and 3 will allow you to specify date in any of the three formats. ---
**Author:** ![sharadha\_s](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharadha_s/32/61664_2.png) [@sharadha\_s](https://discuss.elastic.co/u/sharadha_s)\ **Post date:** [July 11, 2020, 5:50am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/9 "2020-07-11T05:50:10Z")
thanks @Vinayak_Sapre . Is this ordering changed in 7.1? As it used to work for us in previous releases. ---
**Author:** ![Vinayak\_Sapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinayak_sapre/32/45939_2.png) [@Vinayak\_Sapre](https://discuss.elastic.co/u/Vinayak_Sapre)\ **Post date:** [July 11, 2020, 6:48am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/10 "2020-07-11T06:48:51Z")
@sharadha_s ES switched from Joda time to Java time in 7.0. [https://www.elastic.co/guide/en/elasticsearch/reference/7.x/migrate-to-java-time.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/migrate-to-java-time.html) Not sure if this is the root cause of your issue because sample patterns in the guide do not align with my theory. I strongly recommend validating all types of dates. ---
**Author:** ![sharadha\_s](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharadha_s/32/61664_2.png) [@sharadha\_s](https://discuss.elastic.co/u/sharadha_s)\ **Post date:** [July 15, 2020, 1:02am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/11 "2020-07-15T01:02:12Z")
yes it totally make sense to me. Thanks for sharing it!! really appreciate it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 12, 2020, 1:02am UTC](https://discuss.elastic.co/t/date-parsing-error-in-elasticsearch-7-1/240607/12 "2020-08-12T01:02:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.