Uniform Time Formats

[mvenkat_in (Venkatesh)]

I have the following 2 fields with different formats in the input.

End Time: 2019-07-26 00:00:05,580
Start Time: 2019-07-26T00:00:02.678+02:100:

I want them to be formatted to a uniform timestamp say YYYY-MM-dd HH:mm:ss,SSS.

The objective is to to have uniform format and also to calculate the time difference in milli seconds.

Kindly suggest.

Thanks

[Shaoranlaos (Christian Stockhaus)]

you can use the date filter for that
https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-date.html

You would have to build a parsing pattern for match. e.g. "yyyy-MM-dd HH:mm:ss,SSS" for the first pattern

example if the format is different in the same field:

date {
  match => [ "<fieldname>", "<format1>", "<format2>" ]
  target => "<fieldname where it should be a UTC timstamp>"
}

[mvenkat_in (Venkatesh)]

Can you please suggest what should be the pattern for
Start Time: 2019-07-26T00:00:02.678+02:100:

I have problem especially with this.

Thanks

[Shaoranlaos (Christian Stockhaus)]

i am not really sure what the last part is (the ":100:")

The beginning would be: yyyy-MM-ddTHH:mm:ss.SSS
All after the millisecond is non standard and i think it will have to be transformed with a ruby filter. (The only way i can think of)

[mvenkat_in (Venkatesh)]

Input:
req_dtm - 2019-07-26T00:00:16.266+02:00
Date Filter to change format
date {
match => [ "req_dtm", "yyyy-MM-dd'T'HH:mm:ss.SSS" ]
target => "request_dtm"
}
Output:
_dateparsefailure

Input:
trans_dtm - 2019-07-26 00:00:16,562
Date Filter to change format
date {
match => [ "trans_dtm", "yyyy-MM-dd HH:mm:ss,SSS" ]
target => "response_dtm"
}
Output:
2019-07-26T00:00:16.562Z -- Looks OK.

Kindly suggest how to handle the first one which is resulting in parsing failure.

Thanks

[Shaoranlaos (Christian Stockhaus)]

now that thems more like a standard dateformat
try to append a ZZ to the first pattern

ZZ  Timezone offset structured as HH:mm (colon in between hour and minute offsets). Example: -07:00.