Hi ,
I've recently upgraded logstash from V7.6.0 to V7.16.3. and have seen some odd behaviors of Grok filter since then.
filter {
grok {
patterns_dir => "C:\software\ELK\config\patterns"
match=> { "message" => "%{CFLOG}" }
add_tag => "has_traceback"
tag_on_failure=>[]
break_on_match=>false
match=> { "uri_stem" =>"%{REQUESTURL}"}
match=> {"uri_stem" =>"%{DATATYPE}"}
match=> {"atime" => "%{STIME}"}
CFLOG ^%{CFDATE:adate}\s+%{TIME:atime}\s+%{NOTSPACE:x_edge_location}\s+%{NUMBER:bytes}\s+%{IP:clientip}\s+%{WORD:verb}\s+%{NOTSPACE}\s+%{NOTSPACE:uri_stem}\s+%{NUMBER:response}\s+%{NOTSPACE:referer}\s+%{NOTSPACE:agent}\s+%{NOTSPACE:uri_query}\s+%{NOTSPACE:cookies}\s+%{NOTSPACE:x_edge_result_type}\s+%{NOTSPACE}\s+%{NOTSPACE}\s+%{NOTSPACE:cs_protocol}\s+%{NUMBER:cs_bytes}\s+%{NOTSPACE:time_taken}\s+%{NOTSPACE:x_forwarded_for}\s+%{NOTSPACE:ssl_protocol}\s+%{NOTSPACE:ssl_cipher}\s+%{WORD:x_edge_response_result_type}\s+%{NOTSPACE:cs_protocol_version}
REQUESTURL /(%{URLFRAGMENT:lang}/)?%{LEGTYPE:legtype}/%{LEGYEAR:legyear}/%{URLFRAGMENT:legnum}?%{GREEDYDATA:legrest}
it appears that above rules worked for all versions prior to V.7.10.0. However, the rule match=> { "uri_stem" =>"%{REQUESTURL}"} failed to match and returned empty matches in any version between V7.10.0 and V7.16.3.
I am wondering if there was any breaking changes of Grok syntax at V7.10.0?
I also tried grok rules in Kibana's grok debugger (V7.16.3), all works fine.
Regards,
Landong