Date filter failing to set logs time as @timestamp field

(EitanVes) #1

Hi Guys,
I have some issue with passing these time stamps from my log files through the Date filter.
13 AUG 2015 | 05:08:57 |

i wrote a grok filter that generates the field "logTimeStamp" as follows:

add_field => { "logTimeStamp" => "%{day} %{month} %{year} %{time}" }
And later a Date filter as follows:

 match =>["logTimeStamp","dd MMM YYYY HH:mm:ss" ]

Yet the Date filter keeps failing an generating error msgs.

any idea why?

Thanks for the support

Timestamp month name date matching
(Magnus Bäck) #2

So... logTimeStamp would in your example contain "13 AUG 2015 05:08:57"? What error message do you get? I think the date parsing library can be picky about the case of month names, so my guess is that it's choking on AUG.

(EitanVes) #3

THanks Magnus,

Error msg is as follows:

[33mFailed parsing date from field {:field=>"logTimeStamp", :value=>"%{day} %{month} %{year} %{time}", :exception=>"Invalid format: \"%{day} %{month} %{year} %{time}\"", :config_parsers=>"dd MMM YYYY HH:mm:ss", :config_locale=>"default=en_US", :level=>:warn}←[0m

my prime suspect is upper case AUG,
i was thinking about either looking for a locale with uppercase writing (failed so far) or altering the month name string to "Aug" or "aug"
in this case, how is it achievable?
What would you suggest?


(Magnus Bäck) #4

It looks like your day, month, year, and time fields are never set, so logTimeStamp gets a bogus value. How are you setting those fields and what does a complete message look like? Use a stdout output with codec => rubydebug.

(system) #5