Logstash: Exclude separator lines in logs

Elastic Stack Logstash
1

We have several applications, which will frequently insert separator lines into logs. I.e., the log entry looks like:

2019-04-02 06:20:17,453 method_name INFO ==========================================================

I use the following grok pattern to parse the line:

%{TIMESTAMP_ISO8601:timestamp} +(?<method>.*) +%{LOGLEVEL:log.level} (?<message>.*)

which generates the following entry in the Grok Debugger:

{
  "method": "method_name",
  "log": {
    "level": "INFO"
  },
  "message": "==========================================================",
  "timestamp": "2019-04-02 00:00:00,000"
}

I would like to drop this and similar entries from the log, when shipping to ES. However, the issue is that the length of this line and the specific character used varies, so what I would like to do is drop any log entry, where the content of the message field is a single, repeated character.

2 
    ruby {
        code => '
            if event.get("message").chars.uniq.length == 1
                event.cancel
            end
        '
    }
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.