Hi there, i just begin using ELK 6.0.0 and i m very happy with.
I would like to minitor my webserver and i installed filebeat.
Logstash receives informations from webserver and kibana displays it.
Here is kibana display :
{
"_index": "filebeat-2018.05.12",
"_type": "doc",
"_id": "9JwkVGMBKxo6fAmiLZJC",
"_version": 1,
"_score": null,
"_source": {
"@timestamp": "2018-05-12T11:37:21.523Z",
"offset": 37404,
"@version": "1",
"beat": {
"name": "WINDEV",
"hostname": "WINDEV",
"version": "6.0.0"
},
"host": "WINDEV",
"source": "c:\inetpub\logs\LogFiles\W3SVC\u_ex180512.log",
"message": "2018-05-12 11:36:33 W3SVC1 WINDEV 192.168.1.14 GET /atlas.ui/ - 80 - 90.107.XXX.XXX HTTP/1.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.139+Safari/537.36 - - 90.107.XXX.XXX 200 0 121 0 412 32748",
"fields": {
"windev_environment": "production"
},
"tags": [
"ServeurWeb",
"beats_input_codec_plain_applied"
]
},
"fields": {
"@timestamp": [
"2018-05-12T11:37:21.523Z"
]
},
"sort": [
1526125041523
]
}
and here is my logstash filter conf
if [type] == "iis_log" {
if [message] =~ "^#" {
drop {}
}
grok {
match => [ "message", "%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE:s-sitename} %{WORD:cs-method} %{URIPATH:cs-uri-stem} %{NOTSPACE:cs-uri-query} %{NUMBER:s-port} %{NOTSPACE:cs-username} %{IPORHOST:c-ip} %{NOTSPACE:cs(User-Agent)} %{NOTSPACE:cs(Cookie)} %{NOTSPACE:cs(Referer)} %{NOTSPACE:cs-host} %{NUMBER:sc-status:int} %{NUMBER:sc-substatus:int} %{NUMBER:sc-win32-status:int} %{NUMBER:sc-bytes:int} %{NUMBER:cs-bytes:int} %{NUMBER:time-taken:int}" ,
"message", "%{TIMESTAMP_ISO8601:timestamp} %{IPORHOST:s-sitename} %{WORD:cs-method} %{URIPATH:cs-uri-stem} %{NOTSPACE:cs-uri-query} %{NUMBER:s-port} %{NOTSPACE:cs-username} %{IPORHOST:c-ip} %{NOTSPACE:cs(User-Agent)} %{NOTSPACE:cs(Referer)} %{NUMBER:response:int} %{NUMBER:sc-substatus:int} %{NUMBER:sc-substatus:int} %{NUMBER:time-taken:int}" ,
"message", "%{TIMESTAMP_ISO8601:timestamp} %{WORD:cs-method} %{URIPATH:cs-uri-stem} %{NOTSPACE:cs-post-data} %{NUMBER:s-port} %{IPORHOST:c-ip} HTTP/%{NUMBER:c-http-version} %{NOTSPACE:cs(User-Agent)} %{NOTSPACE:cs(Cookie)} %{NOTSPACE:cs(Referer)} %{NOTSPACE:cs-host} %{NUMBER:sc-status:int} %{NUMBER:sc-bytes:int} %{NUMBER:cs-bytes:int} %{NUMBER:time-taken:int}"]
}
geoip {
source => "c-ip"
target => "geoip"
add_tag => [ "iis-geoip" ]
}
useragent {
source => "cs(User-Agent)"
}
}
Kibana does not display IIS fields and logstash does not transform ip in geo-point.
Can you help????
Thanks