Date format error from logstash

Elastic Stack Logstash
1

Hello,
I'm getting a date format error from logstash but the format seems to match the posted date object. Can some one see specifically what it is complaining about?

Logstash error:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse field [date] of type [date] in document with id '92KqbncBqhj6kAxaJLWB'. Preview of field's value: '2021-04-02T07:18:45.000Z'"}],"type":"mapper_parsing_exception","reason":"failed to parse field [date] of type [date] in document with id '92KqbncBqhj6kAxaJLWB'. Preview of field's value: '2021-04-02T07:18:45.000Z'","caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [2021-04-02T07:18:45.000Z] with format [yyyy-MM-dd HH:mm:ss,SSS||yyyy/MM/dd HH:mm:ss||yyyy-MM-dd'T'HH:mm:ss.SSSZZZ||yyyy-MM-dd||epoch_millis]","caused_by":{"type":"date_time_parse_exception","reason":"Failed to parse with all enclosed parsers"}}},"status":400}

Here is the document I'm posting:

{
       "message":"2021/02/04 07:18:45 | xxxxxxxx.xxxxxxxx.PDF.1612451907436 INFO  com.ixiasoft.outputgenerator.preprocessor.AntProcessor - Done Processing AntProcessor. wkdy.dita2pdf",
     "exception":"Done Processing AntProcessor. wkdy.dita2pdf",
      "loglevel":"INFO",
          "host.name":"logstash-dev-poc",
       "service":"xxxxxxxx.xxxxxxxx.PDF.1612451907436",
    "@timestamp":"2021-02-04T19:44:07.822Z",
        "logger":"com.ixiasoft.outputgenerator.preprocessor.AntProcessor",
          "date":"2021-04-02T07:18:45.000Z"
}

It seems "date" should match on this format string?
"yyyy-MM-dd'T'HH:mm:ss.SSSZ"

Here is the "date" index mapping:

        "date" : {
          "format" : "yyyy-MM-dd'T'HH:mm:ss.SSSZ||yyyy-MM-dd HH:mm:ss,SSS||yyyy/MM/dd HH:mm:ss||yyyy-MM-dd'T'HH:mm:ss.SSSZZZ||yyyy-MM-dd||epoch_millis",
          "type" : "date",
          "fields" : {
            "keyword" : {
              "ignore_above" : 256,
              "type" : "keyword"
            }
          }
        },

TIA!

2

This is really an elasticsearch question, you might want to ask it in that forum.

The elasticsearch error message is "failed to parse date field [2021-04-02T07:18:45.000Z] with format [yyyy-MM-dd HH:mm:ss,SSS||yyyy/MM/dd HH:mm:ss||yyyy-MM-dd'T'HH:mm:ss.SSSZZZ||yyyy-MM-dd||epoch_millis]". That does not include the yyyy-MM-dd'T'HH:mm:ss.SSSZ that you show at the start of the "format" field from the index mapping, it just has the other five formats. Hard to explain how that could happen if the index mapping really looks the way you say it looks.

3

Sorry, I must have copy/paste an old output. I just re-ran the POST and here is the output. It does seem to show the matching format but still fails:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse field [date] of type [date] in document with id 'KGcZb3cBqhj6kAxank_x'. Preview of field's value: '2021-04-02T07:18:45.000Z'"}],"type":"mapper_parsing_exception","reason":"failed to parse field [date] of type [date] in document with id 'KGcZb3cBqhj6kAxank_x'. Preview of field's value: '2021-04-02T07:18:45.000Z'","caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [2021-04-02T07:18:45.000Z] with format [yyyy-MM-dd'T'HH:mm:ss.SSSZ||yyyy-MM-dd HH:mm:ss,SSS||yyyy/MM/dd HH:mm:ss||yyyy-MM-dd'T'HH:mm:ss.SSSZZZ||yyyy-MM-dd||epoch_millis]","caused_by":{"type":"date_time_parse_exception","reason":"Failed to parse with all enclosed parsers"}}},"status":400}
4

Z would match -0700. Try yyyy-MM-dd'T'HH:mm:ss.SSS'Z'

5

Thank you @Badger, the 'Z' solution did work.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.