# How to set different filters for different logs in beats and logstash 6.3.2

**URL:** https://discuss.elastic.co/t/how-to-set-different-filters-for-different-logs-in-beats-and-logstash-6-3-2/147661
**Category:** Beats
**Tags:** filebeat
**Created:** [September 7, 2018, 6:18am UTC](https://discuss.elastic.co/t/how-to-set-different-filters-for-different-logs-in-beats-and-logstash-6-3-2/147661 "2018-09-07T06:18:08Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Anil\_Bind](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@Anil\_Bind](https://discuss.elastic.co/u/Anil_Bind)
#### Post date: [September 7, 2018, 6:18am UTC](https://discuss.elastic.co/t/how-to-set-different-filters-for-different-logs-in-beats-and-logstash-6-3-2/147661/1 "2018-09-07T06:18:08Z")

</div>

Hi,

First of all, I am using version 6.3.2 for all the beats and elk stack.  
I have elk setup on centOS 7 and the node is windows server 2012 r2, I was able to filter IIS logs by using below filter.

filter {

if [@metadata][beat] == "filebeat" {  
{  
grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:log\_timestamp} %{WORD:S-SiteName} %{NOTSPACE:S-ComputerName} %{IPORHOST:S-IP} %{WORD:CS-Method} %{URIPATH:CS-URI-Stem} (?:-|"%{URIPATH:CS-URI-Query}") %{NUMBER:S-Port} %{NOTSPACE:CS-Username} %{IPORHOST:C-IP} %{NOTSPACE:CS-Version} %{NOTSPACE:CS-UserAgent} %{NOTSPACE:CS-Cookie} %{NOTSPACE:CS-Referer} %{NOTSPACE:CS-Host} %{NUMBER:SC-Status} %{NUMBER:SC-SubStatus} %{NUMBER:SC-Win32-Status} %{NUMBER:SC-Bytes} %{NUMBER:CS-Bytes} %{NUMBER:Time-Taken}"}  
}  
}  
}  
}

The issue is I have a path having custom logs, I have grok pattern also for it, but IDK how to add that filter with the above existing filter  
In the filebeat.yml I have added the path of the log but it uses the pattern of the iss.  
ANY HELP!

PS. I have not used iis module because it does not works with iis 8.5.

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [September 7, 2018, 8:16am UTC](https://discuss.elastic.co/t/how-to-set-different-filters-for-different-logs-in-beats-and-logstash-6-3-2/147661/2 "2018-09-07T08:16:13Z")

</div>

To parse your custom logs you need to add a new grok filter after the existing one.

```auto
if [@metadata][beat] == "filebeat" {
    grok {
        match => { "message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:S-SiteName} {NOTSPACE:S-ComputerName} %{IPORHOST:S-IP} %{WORD:CS-Method} %{URIPATH:CS-URI-Stem} (?:-|"%{URIPATH:CS-URI-Query}") %{NUMBER:S-Port} %{NOTSPACE:CS-Username} %{IPORHOST:C-IP} %{NOTSPACE:CS-Version} %{NOTSPACE:CS-UserAgent} %{NOTSPACE:CS-Cookie} %{NOTSPACE:CS-Referer} %{NOTSPACE:CS-Host} %{NUMBER:SC-Status} %{NUMBER:SC-SubStatus} %{NUMBER:SC-Win32-Status} %{NUMBER:SC-Bytes} %{NUMBER:CS-Bytes} %{NUMBER:Time-Taken}"}
    }
    grok {
       match => { "message" => "{{ your-custom-pattern }}"}
    }
}

```

Alternatively, you could extend the existing pipeline of IIS module of Filebeat. You need to edit `module/iis/access/ingest/default.json` or `module/iis/error/ingest/default.json` depending on which fileset you need. A new pattern can be added to the list of `processors/grok/patterns`, so Ingest node can match your logs. But this requires you to forward events to Elasticsearch.

Feel free to open a pull request on Github with the pattern. It would be appreciated if you contributed it for IIS 8.5. 😉

---

<div class="post-metadata">

### Author: ![Anil\_Bind](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@Anil\_Bind](https://discuss.elastic.co/u/Anil_Bind)
#### Post date: [September 7, 2018, 8:46am UTC](https://discuss.elastic.co/t/how-to-set-different-filters-for-different-logs-in-beats-and-logstash-6-3-2/147661/3 "2018-09-07T08:46:32Z")

</div>

Thanks for ur help, will contribute in GitHub too.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 5, 2018, 8:46am UTC](https://discuss.elastic.co/t/how-to-set-different-filters-for-different-logs-in-beats-and-logstash-6-3-2/147661/4 "2018-10-05T08:46:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
