COMBINEDAPACHELOG did not add any matched fileds?


(Robert Chen) #1

my logstash filter rule is like:

if [type] == "apache" {
grok {
match => ["message", "%{COMBINEDAPACHELOG}"]
match => ["message", "%{HTTPD20_ERRORLOG}"]
}
}

COMBINEDAPACHELOG did not add any matched fileds, or it did not get _grokparsefailure.
Please see the captured screen:


(Robert Chen) #2

if I replace COMBINEDAPACHELOG with the below it works, why?:

if [type] == "apache" {
grok {
match => ["message", "%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} [%{HTTPDATE:timestamp}] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)"]
}
}


(Magnus Bäck) #3

Please do not post Kibana screenshots. Copy/paste from the JSON tab in Kibana instead, or use a stdout { codec => rubydebug } output in Logstash.

Your log isn't a combined log so COMBINEDAPACHELOG doesn't match and you're getting a _grokparsefailure tag as a result. The grok expression you replaced COMBINEDAPACHELOG with matches common log files which is the format of your log.


(Robert Chen) #4

This are copied from COMBINEDAPACHELOG, this matches, but not the COMBINEDAPACHELOG


(Christian Dahlqvist) #5

You have match specified twice in the same grok block for the same field. I believe the correct way to match multiple patterns against the same field is to configure an array. Try something like this:

grok {
  match => { "message" => ["%{COMBINEDAPACHELOG}", "%{HTTPD20_ERRORLOG}"] }
}

(Magnus Bäck) #6

This are copied from COMBINEDAPACHELOG, this matches, but not the COMBINEDAPACHELOG

Please look more carefully. Your expression is copied from COMMONAPACHELOG:

Here's the definition of COMBINEDAPACHELOG:

Your HTTP log is not in combined format. Over and out.


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.