I am using the GELF Docker driver to send logs to Logstash. It has been working well so far but I am having trouble finding a good way to separate nginx access logs and nginx error logs. I was thinking that I would just add a take for each match like the following:
if [image_name] =~ /nginx/ {
grok {
match => [ "message", "%{INT:status} - %{NOTSPACE:referrer} - %{NOTSPACE:domain} - %{IPORHOST:clientip} - %{QS:request} %{INT:body_bytes_sent} %{QS:http_referer}" ]
add_tag => [ "nginx_access" ]
}
grok {
match => [ "message", "(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER}: \*%{NUMBER:tid} %{GREEDYDATA:errormessage}, client: %{IP:client}" ]
add_tag => [ "nginx_err" ]
}
}
This matches the first pattern, adds the tag but then fails on the second pattern and I end up with a _grokparsefailure
. What's a good way to match and tag access/error logs correctly with GELF?