How to use multiple filters and multiple Grok filters

Elastic Stack Logstash
1

Before adding filter it works fine. But after adding filter, it is not working fine. It is showing an exception. I mentioned that logstash-simple config file below.

  1. Actually i want to use multiple grok filter. How to use it ?

  2. I don't want to use type in grok filter. Is it possible to ignore type , when we are using multiple GROK ?

logstash-simple :

input {
beats
{
port => "5044"
}
}
filter{
kv {
filed_split => ","
source => "msg"
}
mutate {
remove_field => [ "msg" ]
}
if [type] == "A" {
grok {

normal chat message entry

match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{DATA:loglevel} %{DATA:source} %{DATA:node} %{DATA:index} %{DATA:link} %{DATA:colon} %{DATA:url} %{DATA:index} %{DATA:session} %{DATA:colon} %{DATA:userid} %{NOTSPACE:userid}" }
}
} else if [type] == "B" {
grok {

normal chat message entry

match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{DATA:loglevel} %{DATA:source} %{DATA:node} %{DATA:index} %{DATA:link} %{DATA:colon} %{DATA:url} %{DATA:index} %{DATA:session} %{DATA:colon} %{DATA:userid} " }
}
} else {
grok {

normal chat message entry

match => { "message" => "%{GREEDYDATA:message}" }
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
}

2

Question continued in Please validate mutiple GROK filter.

3

Thanks for your reply.

When i validate the GROK filter , it works fine. But my Exact question is, whether the format of mentioning the grok filter inside filter brace is valid or not ? (i.e.), Inside one filter brace, i mentioned two grok filter . whether it is correct or not. Incase if it is wrong, please mention the correct one. Please see the example below.

filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} %{DATA:source} %{DATA:colon} %{GREEDYDATA:message}" }
}
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} %{GREEDYDATA:message}" }
}
}

4

Thanks for your reply .

When i given two grok filter, it is always taking the first grok filter only. Eventhough second filter is correct, the data also related to second grok filter format, but it is taking the first grok filter message only.

  1. Why it is not taking the second grok filter, even though the data is related to that format?
  2. Is there is any other way to mention the format correctly ?

Please look the given format below.

filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} %{DATA:source} %{DATA:colon} %{GREEDYDATA:message}" }
}
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} %{GREEDYDATA:message}" }
}
}

5

I'm ignoring this thread since I've responded in the other thread.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.