# Grok filter for IIS no working
**URL:** https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157
**Category:** Logstash
**Created:** [January 14, 2019, 2:41pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157 "2019-01-14T14:41:22Z")
**Posts on this page:** 19
**Page:** 1
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 14, 2019, 2:41pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/1 "2019-01-14T14:41:22Z")
hey there,
my grok filter for IIS logs is not working on logstash, however it worok on kibana debugger:
input {
beats {
port =\> 5044
type =\> "log"
port=\> 5044
type =\> "iis"
}
filter {
if [type] == "iis" {
grok {
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:log\_timestamp} %{WORD:S-SiteName} %{NOTSPACE:S-ComputerName} %{IPORHOST:S-IP} %{WORD:CS-Method} %{URIPATH:CS-URI-Stem} (?:-|"%{URIPATH:CS-URI-Query}") %{NUMBER:S-Port} %{NOTSPACE:CS-Username} %{IPORHOST:C-IP} %{NOTSPACE:CS-Version} %{NOTSPACE:CS-UserAgent} %{NOTSPACE:CS-Cookie} %{NOTSPACE:CS-Referer} %{NOTSPACE:CS-Host} %{NUMBER:SC-Status} %{NUMBER:SC-SubStatus} %{NUMBER:SC-Win32-Status} %{NUMBER:SC-Bytes} %{NUMBER:CS-Bytes} %{NUMBER:Time-Taken}"}
remove\_field =\> ["message"]
```
}
```
}
}
output {
elasticsearch {
hosts =\> "10.175.142.92:9200"
manage\_template =\> false
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document\_type =\> "%{[@metadata][type]}"
}
}}
in Kibana the logs are showing like this:
{
"\_index": "filebeat-2019.01.14",
"\_type": "doc",
"\_id": "X8LOTGgBdSYGhbv\_YsLO",
"\_version": 1,
"\_score": null,
"\_source": {
"beat": {
"hostname": "U4VMUSPUISITG11",
"name": "U4VMUSPUISITG11",
"version": "6.5.4"
},
"input": {
"type": "log"
},
"host": {
"os": {
"platform": "windows",
"family": "windows",
"build": "9200.22620",
"version": "6.2"
},
"name": "U4VMUSPUISITG11",
"id": "e2b6035c-59e1-49c1-be70-dffd00525749",
"architecture": "x86\_64"
},
"@version": "1",
"offset": 464770,
"@timestamp": "2019-01-14T14:40:19.532Z",
"source": "D:\Logfiles\W3SVC1\u\_ex19011414.log",
"prospector": {
"type": "log"
},
"tags": [
"beats\_input\_codec\_plain\_applied"
],
"type": "log",
"message": "2019-01-14 14:39:47 W3SVC1 U4VMUSPUISITG11 10.160.227.166 GET /Customer/Load a=4002229&p=04&b=3696&ac=&ro=EBD%20-%20Queue%20Sales%20Rep%20Brazil 80 americas\ServiceUspCPNonPrd 10.175.140.244 HTTP/1.1 Mozilla/5.0+(X11;+Linux+x86\_64;+Catchpoint)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/59.0.3071.115+Safari/537.36 - - [u4vmuspuisitg11.olqa.preol.dell.com](http://u4vmuspuisitg11.olqa.preol.dell.com) 302 0 0 716 527 946"
},
"fields": {
"@timestamp": [
"2019-01-14T14:40:19.532Z"
]
},
"sort": [
1547476819532
]
}
i would appreciate some help here!
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 14, 2019, 4:33pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/2 "2019-01-14T16:33:52Z")
You are trying to match (?:-|"%{URIPATH:CS-URI-Query}") to
```
"CS-URI-Query" => "a=4002229&p=04&b=3696&ac=&ro=EBD%20-%20Queue%20Sales%20Rep%20Brazil"
```
That is not a URIPATH. It is not even a URIPARAM (since that needs a leading ?}. Try replacing that whole term with %{NOTSPACE}.
Also, for some items you may want to convert to int. For example
```
%{NUMBER:SC-Bytes:int} %{NUMBER:CS-Bytes:int}
```
Personally I do not convert fields like S-Port or SC-Status to int, since I will never need to do calculations on them (would you ever care about the average port number? 🙂 )
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 14, 2019, 5:09pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/3 "2019-01-14T17:09:11Z")
hey Badger,
thanks for your reply. The issue here is not the filter itself, but why is not being applied in the logs
check this screenshots:
from how the log is showing up in kibana
the filter actually works, in grok debugger, inside Kibana:
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 14, 2019, 6:24pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/4 "2019-01-14T18:24:32Z")
The pattern you have will work if there is no query in the URL. It will not work if there are URI parameters.
It is not getting applied because the event has type "log", not type "iis".
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 14, 2019, 7:25pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/5 "2019-01-14T19:25:29Z")
i see... and Could i change it to 'iis'? because in my filebeat.yml, i have the "document\_type: iis" line and in logstash the type=\> "iis"
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 14, 2019, 8:08pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/6 "2019-01-14T20:08:20Z")
> [@Carlos\_Bodini](#):
>
> input {
> beats {
> port =\> 5044
> type =\> "log"
>
> port=\> 5044
> type =\> "iis"
> }
Is that really what your config looks like?
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 15, 2019, 11:28am UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/7 "2019-01-15T11:28:29Z")
right now is looking like this:
but still the same scenario
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 15, 2019, 3:15pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/8 "2019-01-15T15:15:17Z")
> [@Carlos\_Bodini](#):
>
> "type": "log",
> "message": "2019-01-14 14:39:47
In Kibana that is the type you had before. What does it show now?
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 15, 2019, 4:03pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/9 "2019-01-15T16:03:23Z")
still the same thing,
{
"\_index": "filebeat-2019.01.15",
"\_type": "doc",
"\_id": "zc4\_UmgBdSYGhbv\_GS89",
"\_version": 1,
"\_score": null,
"\_source": {
"input": {
"type": "log"
},
"beat": {
"name": "xxxxxxx",
"version": "6.5.4",
"hostname": "xxxxxxxxxxx"
},
"host": {
"architecture": "x86\_64",
"os": {
"family": "windows",
"platform": "windows",
"build": "9200.22620",
"version": "6.2"
},
"name": "xxxxxxxxx",
"id": "65fd3db9-0bfe-45b5-bd81-bdc27a33af3f"
},
"type": "log",
"@version": "1",
"prospector": {
"type": "log"
},
"message": "2019-01-15 16:01:00 W3SVC1 xxxxxxxxxx10.160.227.164 GET /Content/Themes/base/images/JQGrid/ui-icons\_469bdd\_256x240.png - 80 americas\xxxxxxxxxx 10.175.140.246 HTTP/1.1 Mozilla/5.0+(X11;+Linux+x86\_64;+Catchpoint)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/59.0.3071.115+Safari/537.36 HasAppSupportRole=true [http://xxx.x/Content/Themes/base/JQGrid/jquery.jqgrid-ui.css](http://xxx.x/Content/Themes/base/JQGrid/jquery.jqgrid-ui.css) xxxxxx.x 200 0 0 4639 524 31",
"tags": [
"beats\_input\_codec\_plain\_applied"
],
"source": "D:\logfiles\W3SVC1\u\_ex19011516.log",
"@timestamp": "2019-01-15T16:01:32.435Z",
"offset": 205434
},
"fields": {
"@timestamp": [
"2019-01-15T16:01:32.435Z"
]
},
"sort": [
1547568092435
]
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 15, 2019, 4:41pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/10 "2019-01-15T16:41:36Z")
type on a beat input is ignored if the incoming event already has a type field.
In filebeat, document\_type, which sets \_type, was removed in 6.0. Use a custom field as shown [here](https://discuss.elastic.co/t/document-type-is-being-ignored/109667/2?u=badger).
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 15, 2019, 5:19pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/11 "2019-01-15T17:19:26Z")
yep, now the type is showing as IIS
however the "message" filed still not being grok by my filter, maybe is something wrong with my beats.conf?
```
input {
beats {
port => 5044
type => "iis"
}}
filter {
if [type] == "iis" {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:S-SiteName} %{NOTSPACE:S-ComputerName} %{IPORHOST:S-IP} %{WORD:CS-Method} %{URIPATH:CS-URI-Stem} (?:-|\"%{URIPATH:CS-URI-Query}\") %{NUMBER:S-Port} %{NOTSPACE:CS-Username} %{IPORHOST:C-IP} %{NOTSPACE:CS-Version} %{NOTSPACE:CS-UserAgent} %{NOTSPACE:CS-Cookie} %{NOTSPACE:CS-Referer} %{NOTSPACE:CS-Host} %{NUMBER:SC-Status} %{NUMBER:SC-SubStatus} %{NUMBER:SC-Win32-Status} %{NUMBER:SC-Bytes} %{NUMBER:CS-Bytes} %{NUMBER:Time-Taken}"}
}
}
}
output {
elasticsearch {
hosts => "10.175.142.92:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 15, 2019, 5:21pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/12 "2019-01-15T17:21:05Z")
Are you getting a \_grokparsefailure tag?
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 15, 2019, 5:25pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/13 "2019-01-15T17:25:13Z")
actually not, no error messages
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 15, 2019, 5:35pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/14 "2019-01-15T17:35:48Z")
Enable debug logging. Do you see a line like
```
[DEBUG][logstash.filters.grok] Running grok filter {:event=>#}
```
If not, then your [type] == "iis" still is not matching. But I cannot think of anything else to explain why.
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 15, 2019, 6:03pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/15 "2019-01-15T18:03:26Z")
there's no [DEBUG][logstash.filters.grok] errors in the log file, it seams the filter is not being triggered by some reason
---
### Author:  [@Rahul07](https://discuss.elastic.co/u/Rahul07)
#### Post date: [January 23, 2019, 4:56pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/16 "2019-01-23T16:56:59Z")
Have you got the solution?
---
### Author:  [@Carlos\_Bodini](https://discuss.elastic.co/u/Carlos_Bodini)
#### Post date: [January 23, 2019, 5:08pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/17 "2019-01-23T17:08:52Z")
not yet
---
### Author:  [@Rahul07](https://discuss.elastic.co/u/Rahul07)
#### Post date: [January 23, 2019, 7:23pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/18 "2019-01-23T19:23:35Z")
Actually i was also facing same issue, with above discussion i got solution.
Can you remove type
input{
file
{ path =\> "/tmp/\*\*\***.log"
start\_position =\>"beginning"
sincedb\_path =\> "NUL"
}
}
filter{
#if[type]=="iis\_log"
#{
grok
{
}
}
output
{
elasticsearch
{ hosts =\> ["localhost:9200"]
index =\> ["**"]
# document\_type =\> "iis\_log"
}
stdout{}
}
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 20, 2019, 7:23pm UTC](https://discuss.elastic.co/t/grok-filter-for-iis-no-working/164157/19 "2019-02-20T19:23:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.