So I created a new grok... you should double check it... you can add this through the UI ( I think that UI was in 7.10.. you should upgrade!) ... or through the API. You can add it to the existing pipeline or you can create your own by copying it to a new name... if you do that see the bottom.
You should learn how to do it... I just lined up the definition with the other grok pattern and rearranged..
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-user.name c-ip cs-version cs(User- agent) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
%{TIMESTAMP_ISO8601:iis.access.time} (?:-|%{NOTSPACE:iis.access.site_name}) (?:-|%{NOTSPACE:iis.access.server_name}) (?:-|%{IPORHOST:source.address}) (?:-|%{WORD:http.request.method}) (?:-|%{NOTSPACE:_tmp.url_path}) (?:-|%{NOTSPACE:_tmp.url_query}) (?:-|%{NUMBER:destination.port:long}) (?:-|%{NOTSPACE:user.name}) (?:-|%{IPORHOST:source.address}) (?:-|HTTP/%{NUMBER:http.version}) (?:-|%{NOTSPACE:user_agent.original}) (?:-|%{NOTSPACE:http.request.referrer}) (?:-|%{NOTSPACE:destination.domain}) (?:-|%{NUMBER:http.response.status_code:long}) (?:-|%{NUMBER:iis.access.sub_status:long}) (?:-|%{NUMBER:iis.access.win32_status:long}) (?:-|%{NUMBER:http.response.body.bytes:long}) (?:-|%{NUMBER:http.request.body.bytes:long}) (?:-|%{NUMBER:temp.duration:long})
After I added it as the top grok pattern
POST /_ingest/pipeline/filebeat-7.14.1-iis-access-pipeline/_simulate
{
"docs": [
{
"_index": "m-index",
"_id": "kMpUTHoBr7SFhhL5-98P",
"_source": {
"@timestamp" : "2021-11-09T03:17:20.0516241Z",
"message": "2021-10-24 22:00:00 W3SVC1 XXCIQWBPD03 151.xxx.154.79 POST /xxdotnet/Login/LoginInfo.asmx/GetExpenseCode - 8080 - 151.001.154.79 HTTP/1.1 Mozilla/4.0+(compatible;+Win32;+WinHttp.WinHttpRequest.5) - 151.010.154.79:8080 200 0 0 452 428 10" }
}
]
}
Results
{
"docs" : [
{
"doc" : {
"_index" : "m-index",
"_type" : "_doc",
"_id" : "kMpUTHoBr7SFhhL5-98P",
"_source" : {
"temp" : { },
"destination" : {
"port" : 8080,
"domain" : "151.010.154.79:8080"
},
"source" : {
"address" : "151.xxx.154.79",
"ip" : "151.xxx.154.79"
},
"error" : {
"message" : "'151.xxx.154.79' is not an IP string literal."
},
"url" : {
"path" : "/xxdotnet/Login/LoginInfo.asmx/GetExpenseCode",
"extension" : "asmx/GetExpenseCode",
"original" : "/xxdotnet/Login/LoginInfo.asmx/GetExpenseCode",
"scheme" : null,
"domain" : "151.010.154.79:8080"
},
"iis" : {
"access" : {
"site_name" : "W3SVC1",
"server_name" : "XXCIQWBPD03",
"sub_status" : 0,
"win32_status" : 0
}
},
"@timestamp" : "2021-10-24T22:00:00.000Z",
"http" : {
"request" : {
"method" : "POST",
"body" : {
"bytes" : 428
}
},
"version" : "1.1",
"response" : {
"body" : {
"bytes" : 452
},
"status_code" : 200
}
},
"event" : {
"duration" : 10000000,
"ingested" : "2021-11-09T03:20:38.642371800Z",
"original" : "2021-10-24 22:00:00 W3SVC1 XXCIQWBPD03 151.xxx.154.79 POST /xxdotnet/Login/LoginInfo.asmx/GetExpenseCode - 8080 - 151.001.154.79 HTTP/1.1 Mozilla/4.0+(compatible;+Win32;+WinHttp.WinHttpRequest.5) - 151.010.154.79:8080 200 0 0 452 428 10",
"created" : "2021-11-09T03:17:20.0516241Z"
},
"user_agent" : {
"name" : "Other",
"original" : "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)",
"os" : {
"name" : "Windows",
"version" : "95",
"full" : "Windows 95"
},
"device" : {
"name" : "Spider"
}
}
},
"_ingest" : {
"timestamp" : "2021-11-09T03:20:38.6423718Z"
}
}
}
]
}
If you change the pipeline name you will need to overwrite it.. in the iis.yml file NOT at the output.elasticsearch section in the filebeat.yml file ... as the input overrides that and thus the default pipeline will be used.
As shown
# Access logs
access:
enabled: true
input:
pipeline: my-new-custom-pipeline
Pick of Ingest Pipeline UI, put the new line in and drag it to the top.
