Timezone offset problem

Elastic Stack Logstash
1

I'm having problems setting @timestamp based on the date, time and time zone offset fields from System Center Configuration Manager client logs.

After much trial and error, I have narrowed down the problem to the time zone offset value in the log only having 3 digits instead of 4 (+300 fails, +0300 works)

This is the date filter I'm using for testing:
date { match => [ "message", "MM-dd-yyyy HH:mm:ss.SSSZ" ] }

Here are the lines I'm sending to logstash:
"01-02-2016 01:02:03.456+300"
"01-02-2016 01:02:03.456+0300"

The first one fails with a dateparsefailure, the second one succeeds. Unfortunately, the offsets in the log are all 3-digits instead of 4.

Any suggestions how to handle this case? I did quite a bit of searching and couldn't find a way to format or pad the offset. I considered dropping it altogether, but that would make it harder to correlate with data from other sources.

Dave

2

Perhaps use the mutate filter's gsub option to insert a leading zero if it's missing? I don't recall if gsub support backreferences, but otherwise you can definitely do it with a grok filter.

3

Magnus, thanks for taking a look.

I was unable to get any traction with either gsub or grok, but succeeded with the following ruby code filter:

  ruby {
     code => " event['timezone'] = '%+05d' % event['timezone'] "
}

That reformats the timezone offset from +300 to +0300 which makes the date filter happy.

I'm still a newbie, so this may not be the best approach, but it works.