I've just upgraded from Logstash 2.4.0 to 5.6.8 with the same configuration file. LS 5.6.8 runs for a while and the process takes over 100% CPU. If I restarted LS, it will then consumes 100% CPU again. I'm using quite a lot of grok filters and wonder if this could be the cause, even though the same config ran smoothly on LS 2.4.0.
I'm looking at LS Pipeline stats but wonder how I should interprete the numbers to troubleshoot. For instance, below are stats for one grok
{
"id" : "parse_raw_log",
"events" : {
"duration_in_millis" : 845407,
"in" : 2911008,
"out" : 2911008
},
"matches" : 2911008,
"failures" : 0,
"patterns_per_field" : {
"log" : 1
},
"name" : "grok"
}
What does this duration_in_millis
mean? When will these numbers indicate an issue?
Thanks