Bug ? Elasticsearch 7 failing to parse custom date format

Elastic Stack Elasticsearch
1

Hello,

I am unable to create an index using the following date format using the latest version of elasticsearch

PUT http://localhost:9200/demo HTTP/1.1
Accept: application/json
Content-Type: application/json
Content-Length: 151
Host: localhost:9200

{
  "mappings": {
    "properties": {
      "date": {
        "type":   "date",
        "format": "yyyy-MM-dd HH:mm:ss aa"
      }
    }
  }
}

Below is the error returned

HTTP/1.1 400 Bad Request
content-type: application/json; charset=UTF-8
content-length: 555

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Failed to parse mapping [_doc]: Invalid format: [yyyy-MM-dd HH:mm:ss aa]: Too many pattern letters: a"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [_doc]: Invalid format: [yyyy-MM-dd HH:mm:ss aa]: Too many pattern letters: a","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: [yyyy-MM-dd HH:mm:ss aa]: Too many pattern letters: a","caused_by":{"type":"illegal_argument_exception","reason":"Too many pattern letters: a"}}},"status":400}

A sample value for the above format would be

11/16/2018 7:18:00 PM

The same format worked fine on v6.x, is this some bug in the latest version or is there a different format / syntax we need to switch onto.

Thanks,
Vikas.

2

It looks like only a single a is supported by java - https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html

There's nothing in the docs to suggest this has changed, but I did test between 6.X and 7.X and it was rejected in both.

3

Well it does look like a bug introduced in 7, i could create a mapping using the following request on 6.6.2

PUT http://localhost:9200/demo HTTP/1.1
Accept: application/json
Content-Type: application/json
Content-Length: 166
Host: localhost:9200

{
  "mappings": {
"_doc" : {
    "properties": {
      "date": {
        "type":   "date",
        "format": "yyyy-MM-dd HH:mm:ss aa"
      }
    }
  }
}
}

Response was

HTTP/1.1 200 OK
Warning: 299 Elasticsearch-6.6.2-3bd3e59 "the default number of shards will change from [5] to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you must manage this on the create index request or with an index template" "Wed, 17 Apr 2019 09:12:11 GMT"
content-type: application/json; charset=UTF-8
content-length: 63

{"acknowledged":true,"shards_acknowledged":true,"index":"demo"}

Also, yes it does work if you remove the "a" which would result in an incorrect date format.

4

There was a change in v7 though https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#breaking_70_java_time_changes

5

Maybe this is an issue already https://github.com/elastic/elasticsearch/issues/40913

6

Yeah, I likely missed that.