Dateparse failure When trying to get date from xml file

[kagashe]

Hi all, I'm trying to update the @timestamp field to be the timestamp pulled from my source xml file.
The xml fields are mapped with xpath and all seem to be parsing fine however when I run the date filter below it does not parse and update the @timestamp field, where am I going wrong?

this is datetime format i'm trying to parse which is mapped to the 'time' field by the xml filter.

    2020-02-01 02:28:39.647919 +0000

And this is my date filter which comes after the xml filter

    filter {
      date {
          match => [
               "time",
               "yyyy-MM-dd HH:mm:ss.SSSSSSZ",
               "yyyy-MM-dd HH:mm:ss.SSSSSS Z",
               "yyyy-MM-dd HH:mm:ss.SSS Z",
               "yyyy-MM-dd HH:mm:ss.SSSZ",
               "yyyy-MM-dd HH:mm:ss.SSS",
               "yyyy-MM-dd HH:mm:ss,SSS",
               "yyyy-MM-dd HH:mm:ss"
              ]
         target => "@timestamp"
       }
    }

any help would be really appreciated, this is starting to do my head in.

[adminunix]

Hello,

I think you have a lot of time formats. Try to simplify with this:

date {
          match => [ "time", "yyyy-MM-dd HH:mm:ss.SSS Z", "ISO8601" ]
          target => "@timestamp"
       }

Also, according to elastic guide: https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html

S --> "fraction of a second. Maximum precision is milliseconds ( SSS ). Beyond that, zeroes are appended."

Regards.

[kagashe]

Hi @adminunix I have replaced my config with yours but I'm still getting the dateparse error and @timestamp is not correct, is there anything else you can think of?

[Badger]

If you use

output { stdout { codec => rubydebug } }

what does the [time] field look like?

[kagashe]

Hi @Badger the output is as follows for the time field.

           "time" => [  [0] "2020-02-16 03:25:28.529609 +0000"  ],

[Badger]

OK, it is an array, so you need to reference the first element using

match => [ "[time][0]", ...

[kagashe]

ok @Badger, my config is now as followsbut I'm still getting the dateparse error?

filter {
  date {
    match => [ "[time][0]", "yyyy-MM-dd HH:mm:ss.SSS Z", "ISO8601" ]
    target => "@timestamp"
  }
}

[kagashe]

Ok i fixed it by changing to this

filter {
  date {
    match => [ "[time][0]", "yyyy-MM-dd HH:mm:ss.SSSSSS Z", "ISO8601" ]
    target => "@timestamp"
  }
}