Parse custom Apache2 log

Elastic Stack Logstash
1

Hello, i have an Apache log like this:

443 84.14.49.234 - - [04/Jul/2016:10:11:32 +0200] "GET /ws/1/kpi HTTP/1.1" 200 63 "-" "Mozilla/5.0 (iPhone; CPU **iPhone OS 8_1_3** like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12B466[5a612141d67644a1]"

i would like to extract all bolded elements
I have already this FILTER config:
filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } geoip { source => "clientip" }
But i can't extract certain elements, like:
in "/ws/1/kpi" i need just kpi
model of phone and version
in "Mobile/12B466[5a612141d67644a1]" extract in a bracketed text

I hope you can help me.
Thank you.

2

The easiest is to use additional grok filters to further parse the fields extracted by the COMBINEDAPACHELOG grok pattern. Alternatively, copy the definition of COMBINEDAPACHELOG into the grok filter block of your Logstash configuration and adjust it to suit your needs. This is more work but will be faster (if that matters in your case).

3

Thank you for your answer.

this is my new config:

filter { grok { patterns_dir => ["./patterns"] match => { "message" => "%{NUMBER:port} %{IP:} - - \[%{HTTPDATE:logedtimestamp}\] \"(?:%{WORD:methode} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{TRUEAGENT:trueagent}"} } }
With pattern : (logstash/patterns/trueagent)

TRUEAGENT ([0-9a-f]{16})

But it doesn't match.
When i put it without %{TRUEAGENT:trueagent} that work, but it not work when i add that.

Thanks

4

As your expression is written you require a string matching your TRUEAGENT pattern to directly follow the referred, which isn't the case with the actual strings you want to match against. You need something like this:

... %{QS:referrer} .*\[%{TRUEAGENT:trueagent}\]\"
5

IT WORKS !
Thank you very much for your help !