Date filter fails to convert String value

[glipinski (Grzegorz Lipiński)]

Hi,

My logs are in xml format.
I am using xml filter to parse them and select specific fields.
One of the fields is Time

xpath => [" .//Time/text()", "timestamp"]

I want to use this filed as a base timestamp (the filed will represent when the event occurred)

thats, why I further use date filter to convert String to Date

  date {
      match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
    }

Unfortunatelly convertion fails, _dateparsefailure is present in event tags

Full pipeline:

input {
      beats {
        port => 5044
      }
    }

    filter{
  xml{
    source => "message"
    target => "doc"
    xpath => [
      ".//Time/text()", "timestamp"
    ]
  }
  date {
      match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
    }
}

Example of date

2017-09-19 22:44:44.238

Could you advice what filters/parameters can I use to fix it?

[magnusbaeck (Magnus Bäck)]

Please show an example event produced by Logstash. Copy/paste from Kibana's JSON tab or use a stdout { codec => rubydebug } output. Also, look in the Logstash log. If the date filter fails it'll give you clues about what it chokes on.

[glipinski (Grzegorz Lipiński)]

Hello,

the issue is resolved.
Value captured by xpath was an array.

https://www.elastic.co/guide/en/logstash/current/plugins-filters-xml.html#plugins-filters-xml-xpath

filter{
  xml{
    source => "message"
    target => "doc"
    xpath => [
      ".//Time/text()", "arrayTimestamp"
    ]
  }
	
  mutate {
    add_field => { "timestamp" => "%{arrayTimestamp[0]}"}
    remove_field => [ "arrayTimestamp" ]
  }

  date {
    match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
    timezone => 'Europe/Berlin'
    target => "timestamp"
  }