How to interpret Logstash Pipeline Stats

I've just upgraded from Logstash 2.4.0 to 5.6.8 with the same configuration file. LS 5.6.8 runs for a while and the process takes over 100% CPU. If I restarted LS, it will then consumes 100% CPU again. I'm using quite a lot of grok filters and wonder if this could be the cause, even though the same config ran smoothly on LS 2.4.0.

I'm looking at LS Pipeline stats but wonder how I should interprete the numbers to troubleshoot. For instance, below are stats for one grok

{
        "id" : "parse_raw_log",
        "events" : {
          "duration_in_millis" : 845407,
          "in" : 2911008,
          "out" : 2911008
        },
        "matches" : 2911008,
        "failures" : 0,
        "patterns_per_field" : {
          "log" : 1
        },
        "name" : "grok"
      }

What does this duration_in_millis mean? When will these numbers indicate an issue?

Thanks

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.