# Filter Multiple Patterns

**URL:** <https://discuss.elastic.co/t/filter-multiple-patterns/98016>\
**Category:** Logstash\
**Created:** [August 23, 2017, 7:29am UTC](https://discuss.elastic.co/t/filter-multiple-patterns/98016 "2017-08-23T07:29:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bhuvanesh](https://avatars.discourse-cdn.com/v4/letter/b/6a8cbe/32.png) [@Bhuvanesh](https://discuss.elastic.co/u/Bhuvanesh)\
**Post date:** [August 23, 2017, 7:29am UTC](https://discuss.elastic.co/t/filter-multiple-patterns/98016/1 "2017-08-23T07:29:17Z")

</div>

Hi All,

I have an ELK integrated log server. Normally I get all the messages in the format

%{SYSLOGTIMESTAMP} %{SYSLOGHOST:sysloghost} %{SYSLOGPROG:syslogprog}: %{GREEDYDATA:Message}

I have two level filters where I check eg : if syslogprog == "apache-access" then do further filtering of the messages. This is working fine.

But, now I have added a new log where the format has a small difference,

%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:sysloghost} %{USER:systemuser}: %{SYSLOGPROG:syslogprog} %{GREEDYDATA:Program}

The log I am referring in the format : -

Jul 31 23:39:17 Server-3 reyan: User-Activity root 100.10.56.1 [7372]: 31 Jul 23:39 df -h

See, here the syslogprog field is shifted to another position. Whenever I am adding this new filter configuration logstash throws error in its log.

How to deal with this case ? Please advise.

Regards,  
Bhuvanesh

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 23, 2017, 9:16pm UTC](https://discuss.elastic.co/t/filter-multiple-patterns/98016/2 "2017-08-23T21:16:12Z")

</div>

Grok filters supports multiple expressions (there's an example in the docs). The expressions will be tried one by one in the order you list them. I think it'll work for you if you list the longer expression first so that Logstash only falls back to the more generic expression if the first one doesn't match.

> See, here the syslogprog field is shifted to another position. Whenever I am adding this new filter configuration logstash throws error in its log.

**Always** tell us exactly what you tried and exactly what error you get.

---

<div class="post-metadata">

**Author:** ![Bhuvanesh](https://avatars.discourse-cdn.com/v4/letter/b/6a8cbe/32.png) [@Bhuvanesh](https://discuss.elastic.co/u/Bhuvanesh)\
**Post date:** [August 25, 2017, 7:02am UTC](https://discuss.elastic.co/t/filter-multiple-patterns/98016/3 "2017-08-25T07:02:14Z")

</div>

Hi Magnus,

Thanks for the reply.

My two filter files are as follows :-

File name : 03-apachef.conf

filter {  
grok {  
match =\> {  
"message" =\> "%{SYSLOGTIMESTAMP} %{SYSLOGHOST:sysloghost} %{SYSLOGPROG:syslogprog}: "  
}  
}  
if [syslogprog] == "apache-access" {

grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:sysloghost} %{SYSLOGPROG:syslogprog}: %{COMBINEDAPACHELOG}" }  
}

```
date {
  match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
  remove_field => ["timestamp", "message", "path", "ident", "auth", "logsource"]
}

}

```

}

File name : 04-activity.conf

filter {  
grok {  
match =\> {  
"message" =\> "%{SYSLOGTIMESTAMP} %{SYSLOGHOST:sysloghost} %{SYSLOGUSER:systemuser}: %{SYSLOGPROG:syslogprog} "  
}  
}

if [syslogprog] == "User-Activity" {

grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:sysloghost} %{USER:systemuser}: %{SYSLOGPROG:syslogprog} %{USER:escalatedto} %{IP:sourceip} [%{NUMBER:Number}]: %{GREEDYDATA:Program}" }  
}

}  
}

First filter works fine, But when I add the second filter I get an error like this

[2017-08-25T01:57:35,869][ERROR][logstash.agent] Pipeline aborted due to error {:exception=\>#\<Grok::PatternError: pattern %{SYSLOGUSER:systemuser} not defined\>

Please advise.

Thanks,  
Bhuvanesh

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 25, 2017, 7:58am UTC](https://discuss.elastic.co/t/filter-multiple-patterns/98016/4 "2017-08-25T07:58:01Z")

</div>

Your Logstash and its plugins doesn't support a SYSLOGUSER pattern. Maybe that pattern was introduced in later versions. You can use another pattern instead, e.g. WORD or NOTSPACE.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2017, 7:58am UTC](https://discuss.elastic.co/t/filter-multiple-patterns/98016/5 "2017-09-22T07:58:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.