I have the following Logstash setup.
Logs are piped to Logstash from Logspout via the logspout-logstash adapter, which adds a few fields to log messages, namely the docker.image
field.
I am able to ingest the logs but am having trouble parsing them. I would like to make some filters based on the Docker image field, below I'm trying to parse and match just the nginx
piece out of the full Docker image, which is similar to organization/nginx:tag
.
There is something wrong with my config though because the tag doesn't look like it is being created and the message field doesn't look like it is being parsed either.
Here's what I have in my config so far:
input {
# Logspout UDP input
udp {
port => 5000
type => logspout
codec => json
}
}
filter {
# Nginx access logs
if [docker.image] =~ /nginx/ {
grok {
match => [ "message", "%{IPORHOST:clientip} - - \[%{HTTPDATE:timestamp}\] %{QS:request} %{INT:status} %{INT:body_bytes_sent} %{QS:http_referer} %{QS:agent}" ]
add_tag => [ "nginx" ]
}
}
}
I have a feeling the field is part of the problem but I'm not sure. Is there something happening behind the scenes that I'm missing? Is there a way to step through a filter to see if a log is hitting my filter?