# Unable to parse IIS logs
**URL:** https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073
**Category:** Beats
**Tags:** filebeat
**Created:** [November 16, 2018, 1:49pm UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073 "2018-11-16T13:49:17Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@JacovanZyl](https://discuss.elastic.co/u/JacovanZyl)
#### Post date: [November 16, 2018, 1:49pm UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/1 "2018-11-16T13:49:17Z")
Hi there
I am having some difficulty parsing IIS logs into the discrete fields.
The following tags keeps showing up:
```
"tags": [
"beats_input_codec_plain_applied",
"_grokparsefailure",
"_geoip_lookup_failure"
```
If I test my filter using stdin and stdout rubydebug all works as intended:
```
{
"cs_cookie" => "_ga=GA1.2.1673373544.1536632622;+selectionCriteria.BaseApp.GlobalSelectionCriteria=%7B%22lastUserLanguageCookie%22%3A%22en_gb%22%7D;+_gid=GA1.2.1287340382.1542184575;+token=8f15da6d-818e-422e-bb3d-d1db4a6c4ae0",
"s-sitename" => "W3SVC1",
"cs_referer" => "https://awesomesite.com/",
"cs_status" => "200",
"cs_username" => "-",
"cs_method" => "PUT",
"X-Forwarded-Port" => "443",
"@version" => "1",
"cs-uri-stem" => "/somestem",
"cs_useragent" => "Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/70.0.3538.102+Safari/537.36",
"cs_host" => "awesomesite",
"sc_bytes" => "1790",
"cs_bytes" => "928",
"@timestamp" => 2018-11-15T23:59:55.000Z,
"time_taken" => "46",
"X-Forwarded-Proto" => "https",
"geoip" => {
"postal_code" => "1506",
"region_name" => "Gauteng",
"country_name" => "South Africa",
"timezone" => "Africa/Johannesburg",
"location" => {
"lon" => 28.3167,
"lat" => -26.1833
},
"ip" => "externalipwashere",
"longitude" => 28.3167,
"continent_code" => "AF",
"city_name" => "Benoni",
"latitude" => -26.1833,
"country_code3" => "ZA",
"region_code" => "GT",
"country_code2" => "ZA"
},
"s_port" => "80",
"cs_substatus" => "0",
"cs_version" => "HTTP/1.1",
"host" => "somehost",
"cs_uri_query" => "-",
"X-auth" => "-",
"sc_win32_status" => "0",
"s_ip" => "ipwashere",
"s-computername" => "somehostname"
}
```
Setup:
Filebeat 6.5
Logstash 6.4
Amazon\_ES 6.3
Beats config:
```
input {
beats {
port => 5044
}
}
filter {
if [message] =~ "^#" {
drop {}
}
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:s-sitename} %{NOTSPACE:s-computername} %{IPORHOST:s_ip} %{WORD:cs_method} %{URIPATH:cs-uri-stem} %{NOTSPACE:cs_uri_query} %{NUMBER:s_port} %{NOTSPACE:cs_username} %{NOTSPACE:cs_version} %{NOTSPACE:cs_useragent} %{NOTSPACE:cs_cookie} %{NOTSPACE:cs_referer} %{NOTSPACE:cs_host} %{NUMBER:cs_status} %{NUMBER:cs_substatus} %{NUMBER:sc_win32_status} %{NUMBER:sc_bytes} %{NUMBER:cs_bytes} %{NUMBER:time_taken} %{IPORHOST:ClientIP} %{NOTSPACE:X-Forwarded-Proto} %{NUMBER:X-Forwarded-Port} %{NOTSPACE:X-auth}"
}
remove_field => "message"
}
date {
match => [
"log_timestamp",
"yyyy-MM-dd HH:mm:ss"
]
target => "@timestamp"
remove_field => "log_timestamp"
}
geoip {
source => "ClientIP"
target => "geoip"
remove_field => "ClientIP"
}
}
output {
amazon_es {
hosts => ["myendpoint"]
region => "myregion"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
```
Any help will be greatly appreciated
---
### Author:  [@bigphil](https://discuss.elastic.co/u/bigphil)
#### Post date: [November 16, 2018, 2:07pm UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/2 "2018-11-16T14:07:47Z")
You should try using the IIS module in Filebeat instead of sending the events to Logstash. Much cleaner and everything is already done for you. You just need to enable the geoip and useragent plugins in Elasticsearch and then enable the module in Filebeat and configure the log paths for the module. You'd then just send the output directly to ES, bypassing Logstash completely, unless you needed any additional processing power from Logstash.
---
### Author:  [@JacovanZyl](https://discuss.elastic.co/u/JacovanZyl)
#### Post date: [November 16, 2018, 2:10pm UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/3 "2018-11-16T14:10:40Z")
Hi Philip
That would have been my first choice but unfortunately Amazon ES does not have the plugin.
[https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-supported-plugins.html](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-supported-plugins.html)
The reason why I am having logstash do the heavy lifting 🙂
Any other suggestions instead of not using aws\_es?
---
### Author:  [@bigphil](https://discuss.elastic.co/u/bigphil)
#### Post date: [November 16, 2018, 2:48pm UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/4 "2018-11-16T14:48:06Z")
Gotcha. Ok, take a look at the IIS module default.json pipeline file and make sure your grok pattern matches what is in there. I think there are multiple patterns that support different versions if IIS. I can't explain why it works with stdin and not beats input though.
---
### Author:  [@JacovanZyl](https://discuss.elastic.co/u/JacovanZyl)
#### Post date: [November 17, 2018, 1:48pm UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/5 "2018-11-17T13:48:46Z")
I checked the pattern and it matches exactly. If I go check the fields in the filebeat index they are all present as well.
I wonder if it might be that it is recognizing the incoming log so the filter fails?
---
### Author:  [@jamesspi](https://discuss.elastic.co/u/jamesspi)
#### Post date: [November 28, 2018, 5:57pm UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/6 "2018-11-28T17:57:27Z")
@JacovanZyl - what template are you using?
Also, do you get the same errors when trying a different output with real data as it's coming in? Like file, etc?
---
### Author:  [@JacovanZyl](https://discuss.elastic.co/u/JacovanZyl)
#### Post date: [December 5, 2018, 11:31am UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/7 "2018-12-05T11:31:36Z")
Hi @jamesspi
Apologies for the late reply re:Invent was amazing!!
I tested with the file output and the filter works beautifully.
This all seems to be pointing to AWS ES being the culprit ☹
I will start up my own ES instance to confirm.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 2, 2019, 11:31am UTC](https://discuss.elastic.co/t/unable-to-parse-iis-logs/157073/8 "2019-01-02T11:31:38Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.