[SOLVED] Forcing a static year to date timestamps

Elastic Stack Logstash
1

Hi,

I'm trying to take a timestamp from a log and force it to use the year 2015 since the log files do not originally include the year. Here's an example log entry:

LOG: 06/02 15:24:03:643 1 Check for updates

In my initial grok I do a %{TIMESTAMP:timestamp} which is a custom pattern of following:

%{MONTHNUM2}\/%{MONTHDAY} %{TIME}

I now need to add year to it. I've tried the following without success even though --config-test doesn't error:

	mutate {
		add_field => { "timestamp" => "2015 %{timestamp}" }
		}
	
	date {
		match => [ "timestamp", "YYYY MM/dd HH:mm:ss:SSS" ]
		}

Debug logs show it does not add the 2015 to the timestamp field:

{:timestamp=>"2016-02-04T07:05:52.443000-0500", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"02/17 14:41:36:848", :exception=>"Invalid format: \"02/17 14:41:36:848\" is malformed at \"/17 14:41:36:848\"", :config_parsers=>"YYYY MM/dd HH:mm:ss:SSS", :config_locale=>"default=en_US", :level=>:warn}

Any ideas? Thanks!

2

Btw, I want to add that the strange thing is that it does appear to be successful, and I see in Kibana the results that everything is forced 2015. However it still tags with grokparsefailure and dateparsefailure and it floods the logs.

3

So what does the timestamp field look like? Output from stdout { codec => rubydebug } would be useful.

(I suspect the problem is that you're using add_field instead of replace.)

4

Gosh, not sure how I missed that one! Thanks! Did the trick just fine.