Hey guys,
I tried setting up json logging in apache, as demonstrated in the logstash book.
Here's the custom access log definition I'm using:
LogFormat "{ \
\"host\":\"beta.jokefire.com.jokefire.com\", \
\"path\":\"/var/log/httpd/jf_beta.jokefire.com_access_log\", \
\"tags\":[\"Jokefire beta.jokefire.com\"], \
\"message\": \"%h %l %u %t \\\"%r\\\" %>s %b\", \
\"timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \
\"clientip\": \"%a\", \
\"duration\": %D, \
\"status\": %>s, \
\"request\": \"%U%q\", \
\"urlpath\": \"%U\", \
\"urlquery\": \"%q\", \
\"method\": \"%m\", \
\"bytes\": %B, \
\"vhost\": \"%v\" \
}" beta.jokefire.com_access_json
CustomLog /var/log/httpd/jf_beta.jokefire.com_access_log beta.jokefire.com_access_json
So my access logs are turning up like this in the actual apache logs on the system:
{ "host":"beta.jokefire.com.jokefire.com", "path":"/var/log/httpd/jf_beta.jokefire.com_access_log", "tags":["Jokefire beta.jokefire.com"], "message": "107.170.139.115 - - [12/Oct/2015:16:35:28 -0400] \"GET /healthcheck.php HTTP/1.1\" 200 5", "timestamp": "2015-10-12T16:35:28-0400", "clientip": "107.170.139.115", "duration": 7995, "status": 200, "request": "/healthcheck.php", "urlpath": "/healthcheck.php", "urlquery": "", "method": "GET", "bytes": 5, "vhost": "beta.jokefire.com" }
On the logstash side I'm attempting to use a simple json filter to capture the results:
filter {
json {
source => jsonmessage
}
}
But the results that are turning up in LS are jsonparsefailures:
{
"_index": "logstash-2015.10.12",
"_type": "apache_beta_error",
"_id": "AVBdvbQMy8hMQNvzYdB5",
"_score": null,
"_source": {
"message": "[Thu Oct 08 18:10:21.537414 2015] [authz_core:debug] [pid 980] mod_authz_core.c(809): [client 107.170.139.115:45943] AH01626: authorization result of Require all granted: granted",
"tags": [
"_jsonparsefailure"
],
"@version": "1",
"@timestamp": "2015-10-12T20:28:43.200Z",
"type": "apache_beta_error",
"file": "/var/log/httpd/jf_beta.jokefire.com_error_log",
"host": "ops1",
"offset": "718959602"
},
"fields": {
"@timestamp": [
1444681723200
]
},
"highlight": {
"type.raw": [
"@kibana-highlighted-field@apache_beta_error@/kibana-highlighted-field@"
],
"type": [
"@kibana-highlighted-field@apache_beta_error@/kibana-highlighted-field@"
]
},
"sort": [
1444681723200
]
}
How can I get rid of the jsonparsefailure errors and get this to parse correctly?
Thanks