I have apache logs like that:
188.122.20.100 [09/Oct/2019:14:00:02 +0200] "POST /something/template/ HTTP/1.1" 200 286 "url.html" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 [FBAN/FBIOS;FBDV/iPhone10,4;FBMD/iPhone;FBSN/iOS;FBSV/13.0;FBSS/2;FBID/phone;FBLC/pl_PL;FBOP/5;FBCR/Orange]" urlagain
And I created GROK pattern which in grok debugger fits perfect:
%{IP:ip}\s\[%{HTTPDATE:timestamp}\]\s\"(?:%{WORD:metoda}\s%{NOTSPACE:request}\s(?:HTTP/%{NUMBER:httpversion}))"\s%{NUMBER:response}\s(?:%{NUMBER:bytes})\s(\"%{GREEDYDATA:URL}\")?\s\"(%{GREEDYDATA:system})?\"\s%{GREEDYDATA:referer}
But in implementation in configuration logstash.conf it did not work in elastic.
Conf file:
input {
beats {
port => 5044
}
}
filter {
grok {
match => [ "message", "%{IP:ip}\s\[%{HTTPDATE:timestamp}\]\s\"(?:%{WORD:metoda}\s%{NOTSPACE:request}\s(?:HTTP/%{NUMBER:httpversion}))"\s%{NUMBER:response}\s(?:%{NUMBER:bytes})\s(\"%{GREEDYDATA:URL}\")?\s\"(%{GREEDYDATA:system})?\"\s%{GREEDYDATA:referer}"
}
}
output {
elasticsearch {
hosts => ["elasticsearch:9200"]
index => "apache-%{+yyyy.MM.dd}"
}