Need more information on logstash

Elastic Stack Logstash
#1

I need easy explanation please :slight_smile: I am a beginner in logstash
i have some questions :slight_smile:

1- What is the tag "_gorkparsefailure" and it used for? if I delete this tag what will happen ?

2- how to remove the tag " multiline " ?

3- how I can make a condition from a field of my grok (condition in grok ) ?

4- "endpoint" what is it ?

thank you in advance

#2

  1. grokparsefailure tag gets added when you use the Grok filter and it was unable to match the input to any of the patterns you have defined. You can change it using the tag_on_failure attribute, including setting it to an empty array if you don't want to add a tag on failure.

  2. I don't think that's a standard tag, check your multiline filter to see if it's being set in there.

  3. Not sure what you mean. You want to check to see if a field exists after you have used Grok? You can do

    if [yourField] {
    ...
    }
    To have a conditional on if a field exists or not.

4 . An endpoint is just a location that something connects to. So Logstash can be an endpoint for another application if that application is sending data to Logstash. Or if Logstash is sending data to Elasticsearch then Elasticsearch is the endpoint for Logstash.

#3

Thank you it's great. I understand better now.
One last question. " Break_on match " used for?

Thanks again.

#4

If you have multiple Grok patterns then Grok will try to match against them in the order which they appear in the config file. As soon as one is matched the filter finishes. break_on_match is set to true by default and this is its behaviour.

However, if you set break_on_match to false then Grok will attempt all patterns no matter what. If the first pattern matches then it will still continue to match all the others you may have defined too. It just allows more flexibility.

As an example, say you had some user data like:

James Bond, MI5 Spy

You could have the following Grok filter to extract all the data you need in one go:

grok {
  match => {
    "message" => [
      "%{GREEDYDATA:full_name}, %{GREEDYDATA:occupation}",
      "%{WORD:first_name} %{WORD:last_name},"
    ]
  }
  break_on_match => false
}

That would pull out full_name, first_name, last_name and occupation all in one Grok as it would do the 2nd pattern even after the first one has matched.

1 Like
#5