Iis error default pipeline -- grok pattern fails for malformed url paths

Elastic Stack Beats
1

I have filebeat installed with the iis module, and it is using the default iis error pipeline for the HTTPERR log files.
A few of the error logs have malformed urls, which the grok patterns don't match.

here's the grok pattern for the filebeat-6.3.0-iis-error-default pipeline:

%{TIMESTAMP_ISO8601:iis.error.time} %{IPORHOST:iis.error.remote_ip} %{NUMBER:iis.error.remote_port} %{IPORHOST:iis.error.server_ip} %{IPORHOST:iis.error.server_port} (?:HTTP/%{NUMBER:iis.error.http_version}|-) (?:%{WORD:iis.error.method}|-) (?:%{URIPATHPARAM:iis.error.url}|-)(?: -)? (?:%{NUMBER:iis.error.response_code}|-) (?:%{NUMBER}|-) (?:%{NOTSPACE:iis.error.reason_phrase}|-) (?:%{NOTSPACE:iis.error.queue_name}|-)

And here are a few examples of the format of the logs that grok failed to parse

2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/0.9 t3 12.2.1 400 - URL -
2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/1.1 GET ./././././../../../../../../../../ 400 - URL -
2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/1.1 GET /..\pixfir~1\how_to_login.html 403 - Forbidden -
2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/1.1 GET ..\..\..\..\..\..\winnt\win.ini 400 - URL -
2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/1.1 GET /�.�./�.�./�.�./�.�./�.�./windows/win.ini 404 - NotFound -
2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/1.1 GET /nessus\..\..\..\..\..\..\winnt\win.ini 403 - Forbidden -
2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/1.1 OPTIONS * 404 - NotFound -
2018-05-05 05:05:55 149.42.83.135 12345 192.168.101.101 443 HTTP/1.1 GET /fee&fie=foe 400 - URL -

Since the grok parser fails at the start of the pipeline, none of the fields get populated, so these logs are not stored properly.

I'm wondering if the part of the grok pattern that looks at the uri should be changed from (?:%{URIPATHPARAM:iis.error.url}|-)to something that captures everything that isn't a space or -?

Maybe (?:(?<iis.error.url>[^\s\-]+)|-)

Thanks

2

Sounds reasonable. Can you open an issue in our Github repository ?

3

Yeah, here is the link to the issue: https://github.com/elastic/beats/issues/7951

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.