# Setting up IIS log collection with filebeat

**URL:** <https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361>\
**Category:** Logstash\
**Created:** [June 18, 2018, 7:20pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361 "2018-06-18T19:20:21Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexserd](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@alexserd](https://discuss.elastic.co/u/alexserd)\
**Post date:** [June 18, 2018, 7:20pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/1 "2018-06-18T19:20:22Z")

</div>

I'm trying to collect IIS logs with filebeat. Here is my filebeat yml file  
How do I configure logstash yml and do I need to create new index in kibana? I'm new to ELK stack any help is appreciated

filebeat.prospectors:

- type: log  
enabled: true  
paths:  
- C:\inetpub\logs\LogFiles\W3SVC1296615932\*  
document\_type: iis\_log  
fields:  
iis: true  
filebeat.config.modules:  
path: ${path.config}/modules.d/\*.yml  
reload.enabled: false  
setup.template.settings:  
index.number\_of\_shards: 3  
setup.kibana:  
output.logstash:  
hosts: ["sna-wsus01:5044"]

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 18, 2018, 9:08pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/2 "2018-06-18T21:08:04Z")

</div>

> How do I configure logstash yml

You probably won't have to touch that file but you do need to set up your Logstash pipeline configuration. See [Installing Logstash | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/beats/libbeat/current/logstash-installation.html).

> and do I need to create new index in kibana?

No, but probably an index _pattern_ that selects the index(es) that you configure Logstash to write to.

---

<div class="post-metadata">

**Author:** ![alexserd](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@alexserd](https://discuss.elastic.co/u/alexserd)\
**Post date:** [June 18, 2018, 9:55pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/3 "2018-06-18T21:55:49Z")

</div>

Magnus, I've created logstash.conf file below and getting this error when trying to run it

PS D:\Elastic\Logstash\bin\> .\logstash.bat -f logstash.conf  
Sending Logstash's logs to D:/Elastic/Logstash/logs which is now configured via log4j2.properties  
[2018-06-18T14:51:01,691][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"D:/Elastic/Logstash/modules/fb\_apache/configuration"}  
[2018-06-18T14:51:01,722][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"D:/Elastic/Logstash/modules/netflow/configuration"}  
[2018-06-18T14:51:01,800][FATAL][logstash.runner] An unexpected error occurred! {:error=\>#\<ArgumentError: Setting "" hasn't been registered\>, :backtrace=\>["D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:37:in `get_setting'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:70:in`set\_value'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:89:in `block in merge'", "org/jruby/RubyHash.java:1343:in`each'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:89:in `merge'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:138:in`va  
lidate\_all'", "D:/Elastic/Logstash/logstash-core/lib/logstash/runner.rb:264:in `execute'", "D:/Elastic/Logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in`run'", "D:/Elastic/Logstash/logstash-core/lib/logstash/runner.rb:219:in `run'", "D:/Elastic/Logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in`run'", "D:\Elastic\Logstash\lib\bootstrap\environment.rb:67:in `'"]}  
[2018-06-18T14:51:01,816][ERROR][org.logstash.Logstash] java.lang.IllegalStateException:org.jruby.exceptions.RaiseException: (SystemExit) exit

LOGSTASH.CONF

## We have IIS configured to use a single log file for all sites

# because logstash can't handle parsing files in different

# directories if they have the same name.

# 

input {  
beats {  
port =\> 5044  
type =\> 'iis'  
}  
}

filter {

## Ignore the comments that IIS will add to the start of the W3C logs

# 

if [message] =~ "^#" {  
drop {}  
}

grok {  
## Very helpful site for building these statements:  
# [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/)  
#  
# This is configured to parse out every field of IIS's W3C format when  
# every field is included in the logs  
#  
match =\> ["message", "%{TIMESTAMP\_ISO8601:log\_timestamp} %{WORD:serviceName} %{WORD:serverName} %{IP:serverIP} %{WORD:method} %{URIPATH:uriStem} %{NOTSPACE:uriQuery} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientIP} %{NOTSPACE:protocolVersion} %{NOTSPACE:userAgent} %{NOTSPACE:cookie} %{NOTSPACE:referer} %{NOTSPACE:requestHost} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"]  
}  
date {  
match =\> ["timestamp", "yyyy-MM-dd HH:mm:ss"]  
locale =\> "en"  
}  
}

# Second filter

filter {  
if "\_grokparsefailure" in [tags] {

```
} else {
# on success remove the message field to save space
mutate {
  remove_field => ["message", "timestamp"]
}

```

}  
}

output {  
elasticsearch {  
hosts =\> ["sna-wsus01:9200"]  
index =\> "logstash-%{+YYYY.MM.dd}"  
template =\> "./conf/logstash-template.json"  
template\_name =\> "logstash"  
document\_type =\> "iis"  
template\_overwrite =\> true  
manage\_template =\> true  
}  
}

---

<div class="post-metadata">

**Author:** ![alexserd](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@alexserd](https://discuss.elastic.co/u/alexserd)\
**Post date:** [June 18, 2018, 10:32pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/4 "2018-06-18T22:32:56Z")

</div>

Also I noticed that the error  
[ERROR][org.logstash.Logstash] java.lang.IllegalStateException:org.jruby.exceptions.RaiseException: (SystemExit) exit is because of write permissions to data folder - so I gave everyone permissions to write but still generating same error. Could be issue with Java ? Server is using Java version 8 update 171

I've tried changing logstash.conf as follows but error persists

input {  
beats {  
port =\> 5044  
}  
}

output {  
elasticsearch {  
hosts =\> ["sna-wsus01:9200"]  
index =\> "logstash-%{+YYYY.MM.dd}"  
template =\> "./conf/logstash-template.json"  
template\_name =\> "logstash"  
document\_type =\> "iis"  
template\_overwrite =\> true  
manage\_template =\> true  
}  
}

---

<div class="post-metadata">

**Author:** ![alexserd](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@alexserd](https://discuss.elastic.co/u/alexserd)\
**Post date:** [June 18, 2018, 10:36pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/5 "2018-06-18T22:36:39Z")

</div>

PS D:\Elastic\Logstash\bin\> .\logstash.bat -f "logstash.conf"  
Sending Logstash's logs to D:/Elastic/Logstash/logs which is now configured via log4j2.properties  
[2018-06-18T15:35:08,871][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"fb\_apache", :directory=\>"D:/Elastic/Logstash/modules/fb\_apache/configuration"}  
[2018-06-18T15:35:08,902][INFO][logstash.modules.scaffold] Initializing module {:module\_name=\>"netflow", :directory=\>"D:/Elastic/Logstash/modules/netflow/configuration"}  
[2018-06-18T15:35:08,981][FATAL][logstash.runner] An unexpected error occurred! {:error=\>#\<ArgumentError: Setting "" hasn't been registered\>, :backtrace=\>["D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:37:in `get_setting'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:70:in`set\_value'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:89:in `block in merge'", "org/jruby/RubyHash.java:1343:in`each'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:89:in `merge'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:138:in`va  
lidate\_all'", "D:/Elastic/Logstash/logstash-core/lib/logstash/runner.rb:264:in `execute'", "D:/Elastic/Logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in`run'", "D:/Elastic/Logstash/logstash-core/lib/logstash/runner.rb:219:in `run'", "D:/Elastic/Logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in`run'  
", "D:\Elastic\Logstash\lib\bootstrap\environment.rb:67:in `'"]}  
[2018-06-18T15:35:09,012][ERROR][org.logstash.Logstash] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 19, 2018, 6:16am UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/6 "2018-06-19T06:16:07Z")

</div>

It looks like it finds something it doesn't like in the settings file. Increasing the log level could give additional clues.

---

<div class="post-metadata">

**Author:** ![alexserd](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@alexserd](https://discuss.elastic.co/u/alexserd)\
**Post date:** [June 19, 2018, 4:16pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/7 "2018-06-19T16:16:44Z")

</div>

Found the issue with the config file

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 17, 2018, 4:27pm UTC](https://discuss.elastic.co/t/setting-up-iis-log-collection-with-filebeat/136361/9 "2018-07-17T16:27:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.