Doesn't work when trying to match on "source" and "message" in the same grok

[Hamish1 (Hamish)]

Good day everyone!

I'm using filebeat and logstash.

Here is the part of logstash filter which is not working as I'm expecting.

if "app_name" in [tags] {

mutate {
    gsub => ["message", "\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]", ""]
  }

  grok {
     match => [
                "source" => "/opt/data/app/logs/%{GREEDYDATA:service_name}/%{GREEDYDATA:log_file_name}",
                "message" => "\[%{GREEDYDATA:severity}\] %{TIMESTAMP_ISO8601:timestamp} - %{GREEDYDATA:message}"
              ]
       overwrite => [ "message" ]
       }

  date {
    match => [ "timestamp", "yyyy-mm-dd HH:mm:ss" ]
    target => "@timestamp"
    timezone => "UTC"
    remove_field => ["timestamp"]
  }
}

As you can see, I'm trying to match and extract values from "source" field and "message" field in the same grok. But works only the first one (in this example - "source").

Is there a way how to achieve that?

Any help will be extremely appreciated!

[Transrian]

By default, Grok stop after the first match successful, which mean that if first parsing is successful (source), message parsing will not be 'invoqued'.

To allow it to match your multiple fields / formats, you will need to disable the option break_on_match