Logstash.yml to collect IIS logs with filebeat error

Elastic Stack Logstash
1

Getting error trying to start logstash with following config file
.\logstash.bat -f logstash.yml

LOGSTASH.YML

input {
beats {
port => 5044
type => 'iis'
}
}

filter {

Ignore the comments that IIS will add to the start of the W3C logs

if [message] =~ "^#" {
drop {}
}

grok {
## Very helpful site for building these statements:
# http://grokdebug.herokuapp.com/
#
# This is configured to parse out every field of IIS's W3C format when
# every field is included in the logs
#
match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:serviceName} %{WORD:serverName} %{IP:serverIP} %{WORD:method} %{URIPATH:uriStem} %{NOTSPACE:uriQuery} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientIP} %{NOTSPACE:protocolVersion} %{NOTSPACE:userAgent} %{NOTSPACE:cookie} %{NOTSPACE:referer} %{NOTSPACE:requestHost} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"]
}
date {
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
locale => "en"
}
}

Second filter

filter {
if "_grokparsefailure" in [tags] {

} else {
# on success remove the message field to save space
mutate {
  remove_field => ["message", "timestamp"]
}

}
}

output {
elasticsearch {
hosts => ["sna-wsus01:9200"]
index => "logstash-%{+YYYY.MM.dd}"
template => "./conf/logstash-template.json"
template_name => "logstash"
document_type => "iis"
template_overwrite => true
manage_template => true
}
}

FILEBEAT.YML

filebeat.prospectors:

  • type: log
    enabled: false
    paths:
    - C:\inetpub\logs\LogFiles\W3SVC1296615932*
    filebeat.config.modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
    setup.template.settings:
    index.number_of_shards: 3
    setup.kibana:
    output.elasticsearch:
    hosts: ["sna-wsus01:5044"]

ERROR

PS D:\Elastic\Logstash\bin> .\logstash.bat
Sending Logstash's logs to D:/Elastic/Logstash/logs which is now configured via log4j2.properties
[2018-06-07T16:43:08,751][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>
"D:/Elastic/Logstash/modules/fb_apache/configuration"}
[2018-06-07T16:43:09,052][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"D
:/Elastic/Logstash/modules/netflow/configuration"}
[2018-06-07T16:43:09,318][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: Sett
ing "" hasn't been registered>, :backtrace=>["D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:37:in get_sett ing'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:70:inset_value'", "D:/Elastic/Logstash/logstash-cor
e/lib/logstash/settings.rb:89:in block in merge'", "org/jruby/RubyHash.java:1343:ineach'", "D:/Elastic/Logstash/logst
ash-core/lib/logstash/settings.rb:89:in merge'", "D:/Elastic/Logstash/logstash-core/lib/logstash/settings.rb:138:inva
lidate_all'", "D:/Elastic/Logstash/logstash-core/lib/logstash/runner.rb:264:in execute'", "D:/Elastic/Logstash/vendor/b undle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:inrun'", "D:/Elastic/Logstash/logstash-core/lib/logstash/ru
nner.rb:219:in run'", "D:/Elastic/Logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:inrun'
", "D:\Elastic\Logstash\lib\bootstrap\environment.rb:67:in `'"]}
[2018-06-07T16:43:09,349][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseE
xception: (SystemExit) exit

2nd question is there a IIS Module that needs to be installed for Logstash?

I was able to setup ELK stack for collecting Windows Event Logs with Winlogbeat but having issues with IIS

Thanks!

2

Seems to be a problem with your logstash.yml. Please post it and make sure you format it as preformatted text using Markdown notation or the </> toolbar button.

2nd question is there a IIS Module that needs to be installed for Logstash?

No.

3

Magnus, the logstash.yml file is above

4

You posted a pipeline configuration file that you for some reason have named logstash.yml. That's the standard name for Logstash's settings file, which is a different file. Logstash isn't starting up because it's having problems loading the settings file, so it's probably getting hold of your pipeline configuration file.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.