Logstash grok problem with elastic

I have apache logs like that:

188.122.20.100 [09/Oct/2019:14:00:02 +0200] "POST /something/template/ HTTP/1.1" 200 286 "url.html" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 [FBAN/FBIOS;FBDV/iPhone10,4;FBMD/iPhone;FBSN/iOS;FBSV/13.0;FBSS/2;FBID/phone;FBLC/pl_PL;FBOP/5;FBCR/Orange]" urlagain

And I created GROK pattern which in grok debugger fits perfect:
%{IP:ip}\s\[%{HTTPDATE:timestamp}\]\s\"(?:%{WORD:metoda}\s%{NOTSPACE:request}\s(?:HTTP/%{NUMBER:httpversion}))"\s%{NUMBER:response}\s(?:%{NUMBER:bytes})\s(\"%{GREEDYDATA:URL}\")?\s\"(%{GREEDYDATA:system})?\"\s%{GREEDYDATA:referer}

But in implementation in configuration logstash.conf it did not work in elastic.
Conf file:

input {
  beats {
    port => 5044
  }
}

filter {
	grok {
         match => [ "message", "%{IP:ip}\s\[%{HTTPDATE:timestamp}\]\s\"(?:%{WORD:metoda}\s%{NOTSPACE:request}\s(?:HTTP/%{NUMBER:httpversion}))"\s%{NUMBER:response}\s(?:%{NUMBER:bytes})\s(\"%{GREEDYDATA:URL}\")?\s\"(%{GREEDYDATA:system})?\"\s%{GREEDYDATA:referer}" 
         }
}

output {
    elasticsearch {
        hosts => ["elasticsearch:9200"]
		index => "apache-%{+yyyy.MM.dd}"
    }

You need to escape the double quotes after httpversion and you need to add ] after the pattern.

1 Like

Thank you, without even getting that, we changed it to be like this:

'%{IP:ip}\s[%{HTTPDATE:timestamp}]\s"(?:%{WORD:metoda}\s%{NOTSPACE:request}\s(?:HTTP/%{NUMBER:httpversion}))"\s%{NUMBER:response}\s(?:%{NUMBER:bytes})\s("%{GREEDYDATA:URL}")?\s"(%{GREEDYDATA:system})?"\s%{GREEDYDATA:referer}'

And it worked as well