How does logstash match?

Elastic Stack Logstash
1

Hello,
I need to match some logs that differ only in one fields (url), I match with grok, each grok rule matches a log, so I have different filters but with different grok rules, but now I realize that some logs after doing the matches with grok they continue towards other filters and fields are added or removed because each game has its own rules, for example:

filter {
  grok {
    match => { "message" => "\"%{TIMESTAMP_ISO8601:timestamp}\"\|\"%{GREEDYDATA:url}path/to/match\"" }
    add_field => [ "service", "01" ]
    add_field => [ "type", "send" ]
  }
  mutate {
    copy => { "timestamp" => "reponse" }
    replace => { "url" => "%{url}path/to/match" }
  }
}

filter {
  grok {
    match => { "message" => "\"%{TIMESTAMP_ISO8601:timestamp}\"\|\"%{GREEDYDATA:url}other/path/to/match\"" }
    add_field => [ "service", "01" ]
    add_field => [ "type", "request" ]
  }
  mutate {
    copy => { "timestamp" => "request" }
    replace => { "url" => "%{url}other/path/to/match" }
  }
}

Now into the document of log that match the first rule i see the field response and request, I expected to see only response, same thing for the url field, I see the fields counted, %{url}path/to/matchother/path/to/match

Is there a way to define that once a grok match is done, the log shouldn't be processed anymore?

Thanks

2

I solved with add_field into grok and with if out of grok

filter {
  grok {
    match => { "message" => "\"%{TIMESTAMP_ISO8601:timestamp}\"\|\"%{GREEDYDATA:url}path/to/match\"" }
    add_field => [ "service", "01" ]
    add_field => [ "type", "send" ]
    add_field => [ "grok_match", "service01-send" ]
  }
  if ["grok_match"] == "service01-send" {
    mutate {
      copy => { "timestamp" => "response" }
      replace => { "url" => "%{url}path/to/match" }
  }
}
3

You could make additional processing conditional on there being a _grokparsefailure tag, but things will scale better if you use a tag for sucess

grok { ... add_tag => [ "grokked" ] }
if "grokked" not in [tags] {
    grok { ... add_tag => [ "grokked" ] }
    # Plus other filters
end
if "grokked" not in [tags] {
    grok { ... add_tag => [ "grokked" ] }
    # Plus other filters
end
if "grokked" not in [tags] {
    grok { ... add_tag => [ "grokked" ] }
    # Plus other filters
end
4

better to add a field or better a tag?

5

I do not think it makes much difference.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.