How many matches, we can have in grok filter - as example below

Elastic Stack Logstash
1

I have assorted logs coming to syslog file as one all.log, i am looking sort using different grok match, is this possible as below : match ?

input {
file {
path => "/var/syslog-ng/raw/*.log"
start_position => "beginning"
type => "logstash-syslog"
tags => [ "logstash-syslog" ]
}
}
filter {
grok {
match => {

                   [ "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{HOSTNAME:device_src} %{NUMBER} %{SYSLOGTIMESTAMP} %{WORD}: %{GREEDYDATA:syslog_message}",
                    "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{HOSTNAME:device_src} %{GREEDYDATA:syslog_message}"]
            }
    }
    date {
             match => ["syslog_timestamp", "MMM dd HH:mm:ss" ]
    }
    ruby {
  code => "event.remove('type')"

}
}

output {
elasticsearch {
hosts => ["192.168.1.75:9200"]
index => "logstash-syslog"
}
stdout {
codec => rubydebug
}
}

2

The documentation of the match option in the grok filter documentation contains an example of how to specify more than one expression.

3

this what i see in the document.

grok {
match => {
"message" => ["expr1", "expr2", ..., "exprN"]
}
}

is that mean, i can have 100 expression ? 30 different logs and log format stored in the same file

Curiosity do i consider delay, processing 100 expressions ?

4

is that mean, i can have 100 expression ? 30 different logs and log format stored in the same file

Yes.

Curiosity do i consider delay, processing 100 expressions ?

Of course, but the grok filter doesn't always test all expressions. It terminates the search as soon as there's a match, so you'll want to sort the expressions in "most likely to match" order.

5

Appreciate your reply.

"most likely to match" order

This means, we should not have any generic expression which consumes more time to recursive lookup.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.