Filter IIS Log information

Elastic Stack Logstash
1

Hello everybody,

i just set up the ELK Stack on an Ubuntu machine. Currently Logstash is processing the IIS Logs with a Grok Filter and Splits the information like it should. Now i am Stuck - i would like to "filter" the logs which are sent to the Logstash Server. Currently there are a lot of /health checks in the IIS log which i really dont need in Logstash. Is it possible to not send these lines of the Log? How could i Check for the "page" Part in the IIS log if it contains /health?

Result should be: send everything except the Loglines which are for the page /health

I couldnt find any fitting solutions around here.

Best regards and thanks in advance!

2

How are you sending the IIS logs to Logstash?

3

the Logs are sent via Filebeat. Sorry for the missing information. if you need any further just let me know.

4

The latest filebeat can exclude events - https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#exclude-lines

5

Hi warkolm,

thank you for the information! I have tried some things and came to the conclusion that i would like to handle it serverside. so i have got the configuration for that in one place. I have found a way via "drop" in the filter to drop the loglines. I just cant get it running. Could you have a look at the filter config what is wrong there?

filter {
if [input_type] == "iis" {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:iisSite} %{IPORHOST:computername} %{IP:sourceip} %{WORD:csmethod} %{URIPATH:page} %{NOTSPACE:querystring} %{NUMBER:port} %{NOTSPACE:csusername} %{IP:cip} %{NOTSPACE:useragent} %{NOTSPACE:referer} %{NOTSPACE:cshost} %{NUMBER:scstatus} %{NUMBER:scsubstatus} %{NUMBER:scwin32status} %{NUMBER:scbytes} %{NUMBER:csbytes} %{NUMBER:timetaken}" }
}
if ([message] =~ "/health/") {
drop {}
}
}
}

when i run the configtest i get following error which i dont know how to handle:

root@stw-ch-log-01:/etc/logstash/conf.d# service logstash configtest
SyntaxError: (eval):162: halth
unknown regexp options - halth
eval at org/jruby/RubyKernel.java:1079
initialize at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/pipeline.rb:57
execute at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/agent.rb:172
run at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/runner.rb:90
call at org/jruby/RubyProc.java:281
run at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.2-java/lib/logstash/runner.rb:95
call at org/jruby/RubyProc.java:281
initialize at /opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:24