IIS Module - not groking

Elastic Stack Beats
1

I have inherited a partially constructed ELK environment. Previous to my efforts, IIS logs have not entered the system. Looking at previous source control checkins, the Dev before me was running everything locally on the ELK server (configs all say localhost).
Server is Ubuntu
Client is Windows running IIS 7.
Logging is set to W3C, here is a sample with ip changed a bit:
2018-10-14 17:22:52 172.17.200.154 GET /Default.aspx ReturnUrl=%2f 443 - 12.0.0.0 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/69.0.3497.100+Safari/537.36 200 0 0 2620

I followed the IIS Module instructions:
Installed the Ingest GEOIP and User Agent.

On the windows IIS box, I installed filebeat first and got that shipping logs, I then installed the IIS module in powershell. I also ran filebeat setup which created numerous dashboards. The dashboards have no data which isn't surprising due to lack of parsing data. In filebeats.yml I commented out logstash and only have elasticsearch and kibana connections.

The filebeats log shows my iis logs being harvested. I can see these in Kabana, but there is a grok error message: "Provided Grok expressions do not match field value:" followed by the row in iis log.

After starting filebeats with the IIS module loaded, I noticed there are duplicate indexes available in Kibana. Both are named "filebeat-*" I had thought it should add a date stamp, but eventually I would want to change that name to something more descriptive.

My theory is that its busted since its going into the same/similar indexes? I am looking at sending the logs to a new index and saw some documentation about that, but I got confused by the templates, I thought filebeats setup would do that for me.

Thanks!

2

Can you share your filebeat configuration?

Which filebeat version are you using?

Which IIS version are you using? Apparently the log format is somewhat different between different IIS versions. There are a number of posts with older IIS versions and fixes/improvements to grok.

3

Thanks for the help, steffens!

I am using filebeat version 6.4.2.
IIS is version 7.

I have also been following this post Parsing problem for iis server error log using filebeat 6.3.2 which I discovered after my post (I tried searching but his didn't come up).

But he stared directly into the abys, and did not survive :slight_smile:

I had read elsewhere that the grok patterns for IIS Module should not be edited. I can't find the source where I read that now. Should I be working with module\iis\error\ingest.json?

4

I edited the ingest.json file and this appears to have fixed most of my issues. The dashboard is still complaining, but I think it is due to other indexes I have saved which it may be trying to access.

Thanks for the help.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.