Date filter failing to set logs time as @timestamp field


(EitanVes) #1

Hi Guys,
I have some issue with passing these time stamps from my log files through the Date filter.
13 AUG 2015 | 05:08:57 |

i wrote a grok filter that generates the field "logTimeStamp" as follows:

mutate{
add_field => { "logTimeStamp" => "%{day} %{month} %{year} %{time}" }
}
And later a Date filter as follows:

date{
 match =>["logTimeStamp","dd MMM YYYY HH:mm:ss" ]
}

Yet the Date filter keeps failing an generating error msgs.

any idea why?

Thanks for the support
Eitan


Timestamp month name date matching
(Magnus Bäck) #2

So... logTimeStamp would in your example contain "13 AUG 2015 05:08:57"? What error message do you get? I think the date parsing library can be picky about the case of month names, so my guess is that it's choking on AUG.


(EitanVes) #3

THanks Magnus,

Error msg is as follows:

[33mFailed parsing date from field {:field=>"logTimeStamp", :value=>"%{day} %{month} %{year} %{time}", :exception=>"Invalid format: \"%{day} %{month} %{year} %{time}\"", :config_parsers=>"dd MMM YYYY HH:mm:ss", :config_locale=>"default=en_US", :level=>:warn}←[0m

my prime suspect is upper case AUG,
i was thinking about either looking for a locale with uppercase writing (failed so far) or altering the month name string to "Aug" or "aug"
in this case, how is it achievable?
What would you suggest?

THanks!
Eitan


(Magnus Bäck) #4

It looks like your day, month, year, and time fields are never set, so logTimeStamp gets a bogus value. How are you setting those fields and what does a complete message look like? Use a stdout output with codec => rubydebug.


(system) #5