Can't use filebeat [beat][timezone] with logstash date filter plugin?

[PSM]

I'm sending a log file to logstash using filebeat that does not encode timezone in the timestamp:

2019-03-08 17:48:49,766 [3] INFO Corp.FO.DataLoaders.BaseDataFile [DOMAIN\username] - log message detail

Until today, when daylight savings changed, I've been seeding the logstash date filter plugin with the [beat][timezone] field. I add this to events using the filebeat processor "add_locale."

FILEBEAT CONFIGURATION

processors:
- add_locale:
    format: abbreviation

LOGSTASH CONFIGURATION

date {
    locale => "en"
    match => ["corp.log.entryTimestamp", "yyyy-MM-dd HH:mm:ss,SSS"]
    timezone => "%{[beat][timezone]}"
    tag_on_failure => ["timestamp_parse_failed"]
    target =>  "@timestamp"
}

When daylight savings started on the east coast of the US today, the [beat][timezone] field was sent 'EDT' instead of 'EST' which logstash couldn't parse because 'EDT' is not a valid canonical ID timezone for logstash (http://joda-time.sourceforge.net/timezones.html). I had the same issue with CEST and CET.

Any idea how to get this working or do I just need to set it manually?

Thanks!

Peter

[Badger]

If you know that in your case an ambiguous timezone name always refers to a particular canonical timezone then you can use mutate+gsub to replace it before giving it to the date filter.

[PSM]

Yeah, that's what I had to do today. However, I need to do this for about 6 regions and it just seems silly to hard code this when the formats should just match between the two, no?

[Badger]

filebeat just returns whatever go returns. It would be nice if that were unambiguous.