Create standard timestamp

[tharu85 (Tharaka)]

In my logstash, I used KV filter to parse the log and now I have 2 fields for date and time.

"date" => "2017-10-10",
"time" => "12:18:23",

But I need to build standard timestamp from those date and time fields as below.

"log_timestamp" => 2017-10-10T12:18:23.324Z,

I tried following syntax under logstash filter and it does not work and gave _dateparsefailure" under tags.

mutate {
      add_field => ["log_timestamp", "%{date} %{time}"]
}
date {
      match => [ "log_timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ" ]
}

gave following error under "tags"

"tags" => [
    [0] "_dateparsefailure"
],

Appreciate if any one guide me to meet my requirement.

[magnusbaeck (Magnus Bäck)]

The timestamp pattern in the date filter is supposed to match the timestamp you want to parse ("2017-10-10 12:18:23"). It doesn't describe the desired output format.

[tharu85 (Tharaka)]

The syntax which I tried have additional parameters. Example mili-seconds and 'T'. What is the purpose of putting 'T' in between date and time. and will i able to set .000 as mili-seconds after time

date {
  match => [ "log_timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ" ]
}

[magnusbaeck (Magnus Bäck)]

What is the purpose of putting 'T' in between date and time.

That's part of the ISO8601 standard.

and will i able to set .000 as mili-seconds after time

If you don't include SSS in your date pattern the milliseconds will default to zero.

So, to be very explicit your date pattern should be: YYYY-MM-dd HH:mm:ss

[tharu85 (Tharaka)]

I have fixed it my self, after tried several syntax

 date {
		match => [ "log_timestamp", "ISO8601", "YYYY-MM-dd HH:mm:ss", "YYYY-MM-dd HH:mm:ss.ZZZ" ]
		target => "log_timestamp"
	}