Logstash filter to create a subfield based on specific text in a log message

Elastic Stack Logstash
1

I've been working on a Logstash configuration where I'm trying to create a subfield within the 'message1' field based on a specific text pattern ('Started'). Here's a snippet of my current Logstash filter:

filter {
  grok {
    pattern_definitions => {
      "Extract_log_line" => "%{TIMESTAMP_ISO8601:timestamp} \| %{LOGLEVEL:log_level} \| \[%{DATA:thread}\] --- %{NUMBER:process_id} \| %{DATA:class} \| \| %{GREEDYDATA:message1}(\\r|\\n)?"
      "Extract_routes_started" => ".*Started %{DATA:routes_started} .*"
    }
    match => {
      "message" => [
        "%{Extract_log_line}"
      ]
    }
    add_field => {
      "[message1][routes_started]" => "%{routes_started}"
    }
  }
  mutate {
    remove_field => ["@timestamp", "@version"]
  }
}

Unfortunately, this doesn't seem to be working as expected "_grokparsefailure" I'm trying to detect the text 'Started' within the 'message1' field and create a subfield 'routes_started' based on the text that follows it. Could anyone assist in identifying what might be incorrect in my Logstash configuration or suggest a better approach to achieve this?

2

You never reference the Extract_routes_started pattern that you defined, so [routes_started] will never exist. I can't say why the Extract_log_line pattern does not match without seeing a sample of [message].

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.