Parse apache access log all the fields and draw the graph using status/response

Elastic Stack Logstash
1

I need to filter status like 200 or response in kibana graph. But available fields like status/response not showing to filter in Discover.

Logstash code :
input {
beats {
port => "5044"
}
}

filter {
if [type] == "log" {
grok {
match => { "message" => '%{IPORHOST:clientip} - - [%{DATA:day}/%{WORD:month}/%{YEAR:year}:%{DATA:hour}:%{DATA:minute}:%{DATA:seconds}] "%{WORD:method} %{DATA:url} HTTP/%{NUMBER:http_version}" %{DATA:status} %{DATA:response} "%{DATA:referurl}" "%{WORD:client}/%{DATA:clinet_version}"'
}
remove_field => "message"
}
}
}

output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}

Ouput in Kibana Discover :

message 192.168.1.199 - - [21/Mar/2019:09:22:23 +0530] "GET / HTTP/1.1" 403 4961 "-" "avast! Antivirus"

But the above log combined into single field, i need a separate field like clientip/
status/resposne

2

You example line does not match your grok Pattern. For example. "avast! Antivirus" does not match "%{WORD:client}/%{DATA:clinet_version}".

Try

match => { "message" => '^%{IPORHOST:clientip} - - \[%{DATA:ts}\] "%{WORD:method} %{DATA:url} HTTP/%{NUMBER:http_version}" %{DATA:status} %{DATA:response} "%{DATA:referurl}" "%{DATA:client}"' }
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.