Logstash parsing and Watcher alerting

Elastic Stack Logstash
9

OK, so we started off with lines like these

Jul 23 09:24:16 mmr mmr-core[4268]: Processing file [Aweg3AOMTs_1563866656876839.mt], passing to thread [5147]
Jul 23 09:24:16 mmr mmr-core[5147]: Aweg3AOMTs_1563866656876839.mt database [SELECT carrier[...]

using dissect to remove the first part, that leaves us with this in the restOfLine field.

Processing file [Aweg3AOMTs_1563866656876839.mt], passing to thread [5147]
 Aweg3AOMTs_1563866656876839.mt database [SELECT carrier[...]

What I did in that grok pattern was to give a list of patterns to try against that field. Note that the patterns are anchored to the start of the field using ^, which means they fail to match very quickly.

If you want to parse the 4 fields from the id I would change that a little

    grok {
        match => {
            "restOfLine" => [
                "^Processing file \[%{NOTSPACE:correlationId}\]",
                "^ %{NOTSPACE:correlationId} "
            ]
        }
    }
    grok {
        pattern_definitions => { "SOMETEXT" => "[[:alnum:]]+" }
        match => {
            "correlationId" => [
                "^%{SOMETEXT:text}_%{INT:num1}\.%{INT:num2}\.%{DATA:suffix}$",
                "^%{SOMETEXT:text}_%{INT:num1}\.%{DATA:suffix}$"
            ]
        }
    }
1 Like