I'm trying to parse the access log from NGINX with Logstash but, I get an _jsonparsefailure although I'm not using a JSON format. Why do I get that _jsonparsefailure?
My input lines are like
nginx: 127.0.0.1 - - [24/Oct/2016:10:34:59 +0200] "GET /app/kibana HTTP/1.1" 200 5018 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"`
My config file in Logstash is:
input {
kafka {
topic_id => "access-log-nginx"
zk_connect => "xxx:2181"
consumer_threads => 4
}
}
filter {
grok{
match=>{
"message"=>"%{WORD:nginx}\: %{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}"
}
}
mutate {
convert => ["response", "integer"]
convert => ["bytes", "integer"]
convert => ["responsetime", "float"]
}
geoip {
source => "clientip"
target => "geoip"
add_tag => [ "nginx-geoip" ]
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["xxx:9200"]
user => "spark-user"
password => "spark-us3r"
index => "access_log_%{+YYYY.MM.dd}"
}
}
The output from logstash:
{
"message" => "nginx: 127.0.0.1 - - [24/Oct/2016:10:34:59 +0200] \"GET /app/kibana HTTP/1.1\" 200 5018 \"-\" \"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2\"",
"tags" => [
**[0] "_jsonparsefailure"**
],
"@version" => "1",
"@timestamp" => "2016-10-24T08:35:05.488Z",
"nginx" => "nginx",
"clientip" => "127.0.0.1",
"ident" => "-",
"auth" => "-",
"timestamp" => "24/Oct/2016:10:34:59 +0200",
"verb" => "GET",
"request" => "/app/kibana",
"httpversion" => "1.1",
"response" => 200,
"bytes" => 5018,
"referrer" => "\"-\"",
"agent" => "\"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2\""
}