Can't use filebeat [beat][timezone] with logstash date filter plugin?

I'm sending a log file to logstash using filebeat that does not encode timezone in the timestamp:

2019-03-08 17:48:49,766 [3] INFO Corp.FO.DataLoaders.BaseDataFile [DOMAIN\username] - log message detail

Until today, when daylight savings changed, I've been seeding the logstash date filter plugin with the [beat][timezone] field. I add this to events using the filebeat processor "add_locale."


- add_locale:
    format: abbreviation


date {
    locale => "en"
    match => ["corp.log.entryTimestamp", "yyyy-MM-dd HH:mm:ss,SSS"]
    timezone => "%{[beat][timezone]}"
    tag_on_failure => ["timestamp_parse_failed"]
    target =>  "@timestamp"

When daylight savings started on the east coast of the US today, the [beat][timezone] field was sent 'EDT' instead of 'EST' which logstash couldn't parse because 'EDT' is not a valid canonical ID timezone for logstash ( I had the same issue with CEST and CET.

Any idea how to get this working or do I just need to set it manually?



If you know that in your case an ambiguous timezone name always refers to a particular canonical timezone then you can use mutate+gsub to replace it before giving it to the date filter.

Yeah, that's what I had to do today. However, I need to do this for about 6 regions and it just seems silly to hard code this when the formats should just match between the two, no?

filebeat just returns whatever go returns. It would be nice if that were unambiguous.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.