Hello,
I'm new to elk and I'm trying to figure out how the grok match keyword works.
What I wonder is if I want to parse different line format, do I have to put multiple grok filters one after the other one and the first match that is ok will be used ? Or do I have to first filter my line format with a test and than apply a match keyword ?
In other word does match work as a pattern-matching selector or not ?
My aim is to analyse logs from different applications which have very different formats,
What you can do is the following:
Image you have 3 different patterns:
filter{
if ("SUCCESS" not in [tags]) {
grok{
match => {"message" => "PATTERN1"}
add_tag => ["SUCCESS"]
remove_tag => ["_grokparsefailure"]
}
if ("SUCCESS" not in [tags]) {
grok{
match => {"message" => "PATTERN2"}
add_tag => ["SUCCESS"]
remove_tag => ["_grokparsefailure"]
}
if ("SUCCESS" not in [tags]) {
grok{
match => {"message" => "PATTERN3"}
add_tag => ["SUCCESS"]
remove_tag => ["_grokparsefailure"]
}
}
And then in ouput, only push if the message does not have a _grokparsefailure tag.
It might also be possible to do this by using multiple patterns in one grok filter and by setting break_on_match (or something like that) to true. But as I don't use it, I cannot guide you for this method. (I personnaly prefer splitting them, it's longer, but easier to read and to split them into multiple .conf files)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.