Replace filter not working properly after upgrade to logstash 7.1.1

Elastic Stack Logstash
1

Hi All,

I am trying to replace the timestamp parsed from my message with existing timestamp + year (parsed from the source of the file).

This below code was working perfectly in logstash 6.7.2 but after upgrading to 7.1.1 its not working anymore.

timestamp = [system][netlog][timestamp]
source = /syslog-data/DC1/NETWORK-DEVICES/UDP/IP/2018/09/IP-2018-09-17.log

=================================CODE==================================
grok {
match => { "source" => "/%{GREEDYDATA}/%{GREEDYDATA}/%{GREEDYDATA}/%{GREEDYDATA}/%{GREEDYDATA}/%{YEAR:year}/%{GREEDYDATA}/%{GREEDYDATA}"}
}

mutate {
replace => [ "[system][netlog][timestamp]", "%{[system][netlog][timestamp]} %{year}" ]
}

date {
match => [
"[system][netlog][timestamp]",
"MMM d HH:mm:ss YYYY",
"MMM dd HH:mm:ss YYYY"
]
target => "@timestamp"
}

mutate {
remove_field => ["year"]
}
==================================CODE END====================================

After this the value of [system][netlog][timestamp] becomes Sep 17 00:00:02 %{year}, but previously it was giving the output Sep 17 00:00:02 2018

Please help and thanks in advance.

2

If you comment out the remove_field does [year] exist?

That grok pattern could be really expensive (not sure if it is expensive enough to timeout). I would change it to

grok { match => { "source" => "/%{YEAR:year}/"} }
3

Nope, your recommended solution didn't work. It just print the %{year} whereas it should put the value of %{year}.

4

If you comment out the remove_field does [year] exist?

5

no, it does't exist. But after a bit of tweaking here and there i found that after replacing the double quotation "%{[system][netlog][timestamp]} %{year}" with single quotation '%{[system][netlog][timestamp]} %{year}' it solved the problem. I don't know know why!! But hey its works. :stuck_out_tongue:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.