i have a log like below
#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2018-08-28 18:24:25
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
2018-08-28 18:24:25 [10.100.220.70](http://10.100.220.70) GET / - 80 - [10.100.118.31](http://10.100.118.31) Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.3;+WOW64;+Trident/7.0;+.NET4.0E;+.NET4.0C;+.NET+CLR+3.5.30729;+.NET+CLR[+2.0.50727](tel:+2050727);+.NET+CLR+3.0.30729) 404 4 2 792
I have installed filebeat and configure the filebeat yml file as below
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
document_type: iis
enabled: false
#============================= Filebeat modules ===============================
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
and i setup iis module as below
- module: iis
access:
enabled: true
var.paths: ["C:/inetpub/logs/LogFiles/*/*.log"]
error:
enabled: true
var.paths: ["C:/Windows/System32/LogFiles/HTTPERR/*.log"]
but i am getting log cannot parse error, what to do ?
steffens
(Steffen Siering)
August 28, 2018, 2:38pm
2
Can you post the complete log message please?
#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2018-08-28 18:24:25
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
2018-08-28 18:24:25 [10.100.220.70](http://10.100.220.70) GET / - 80 - [10.100.118.31](http://10.100.118.31) Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.3;+WOW64;+Trident/7.0;+.NET4.0E;+.NET4.0C;+.NET+CLR+3.5.30729;+.NET+CLR[+2.0.50727](tel:+2050727);+.NET+CLR+3.0.30729) 404 4 2 792
steffens
(Steffen Siering)
August 29, 2018, 7:11pm
4
This is a very different log message. In fact it looks like the beginning of a log file?
Does this log fail as well?
What exactly does the error message say?
I think the filebeat module was tested with IIS 10. Which version are you using? 7.5?
yes ours is iis 7.5. what to do now ?
what about custom log formating using grok? how to do it?
i am attaching the log error getting in kibana
steffens
(Steffen Siering)
August 30, 2018, 2:47pm
7
Unfortunately IIS module does not support overwriting the grok pattern (nginx does by now). Feel free to open a github issue for custom grok patterns, but also IIS 7.5.
You log looks most similar to the IIS grok pattern here:
{
"description": "Pipeline for parsing IIS access logs. Requires the geoip and user_agent plugins.",
"processors": [{
"grok": {
"field": "message",
"patterns":[
"%{TIMESTAMP_ISO8601:iis.access.time} %{IPORHOST:iis.access.server_ip} %{WORD:iis.access.method} %{URIPATH:iis.access.url} %{NOTSPACE:iis.access.query_string} %{NUMBER:iis.access.port} %{NOTSPACE:iis.access.user_name} %{IPORHOST:iis.access.remote_ip} %{NOTSPACE:iis.access.agent} %{NOTSPACE:iis.access.referrer} %{NUMBER:iis.access.response_code} %{NUMBER:iis.access.sub_status} %{NUMBER:iis.access.win32_status} %{NUMBER:iis.access.request_time_ms}",
"%{TIMESTAMP_ISO8601:iis.access.time} %{NOTSPACE:iis.access.site_name} %{WORD:iis.access.method} %{URIPATH:iis.access.url} %{NOTSPACE:iis.access.query_string} %{NUMBER:iis.access.port} %{NOTSPACE:iis.access.user_name} %{IPORHOST:iis.access.remote_ip} %{NOTSPACE:iis.access.agent} %{NOTSPACE:iis.access.cookie} %{NOTSPACE:iis.access.referrer} %{NOTSPACE:iis.access.hostname} %{NUMBER:iis.access.response_code} %{NUMBER:iis.access.sub_status} %{NUMBER:iis.access.win32_status} %{NUMBER:iis.access.body_sent.bytes} %{NUMBER:iis.access.body_received.bytes} %{NUMBER:iis.access.request_time_ms}",
"%{TIMESTAMP_ISO8601:iis.access.time} %{NOTSPACE:iis.access.site_name} %{NOTSPACE:iis.access.server_name} %{IPORHOST:iis.access.server_ip} %{WORD:iis.access.method} %{URIPATH:iis.access.url} %{NOTSPACE:iis.access.query_string} %{NUMBER:iis.access.port} %{NOTSPACE:iis.access.user_name} %{IPORHOST:iis.access.remote_ip} HTTP/%{NUMBER:iis.access.http_version} %{NOTSPACE:iis.access.agent} %{NOTSPACE:iis.access.cookie} %{NOTSPACE:iis.access.referrer} %{NOTSPACE:iis.access.hostname} %{NUMBER:iis.access.response_code} %{NUMBER:iis.access.sub_status} %{NUMBER:iis.access.win32_status} %{NUMBER:iis.access.body_sent.bytes} %{NUMBER:iis.access.body_received.bytes} %{NUMBER:iis.access.request_time_ms}"
],
"ignore_missing": true
}
}, {
"remove":{
"field": "message"
}
}, {
You log is missing the iis.access.referrer
field.
Modules are just config objects. You can try to update the module by adding a new grok pattern and update the ingest node pipeline in your local installation.
sir there is another issue. both s-ip and c-ip values are not in IPHOST format
s-ip =[10.100.220.70](http://10.100.220.70)
c-ip =[10.100.118.31](http://10.100.118.31)
how do i create a grok for it?
i tried a grok by removing referer, still not working because of this ip format
system
(system)
Closed
October 4, 2018, 1:36pm
10
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.