I am having a hard time understanding why my filter to remove some fields is not working.
I grabbed a sample of the JSON from kibana
{
"_index": "logstash-2017.01.27",
"_type": "json",
"_score": 1,
"_source": {
"source": "unknown",
"type": "json",
"duration": 16.13,
"view": 2.68,
"@version": "1",
"host": "ip-10-0-0-111",
"action": "index",
"api": false,
"user_agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",
"controller": "marketing",
"format": "html",
"message": "{"method":"GET","path":"/","format":"html","controller":"marketing","action":"index","status":200,"duration":16.13,"view":2.68,"db":6.11,"route":"marketing#index","parameters":{},"user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36","source":"unknown","tags":["request"],"@timestamp":"2017-01-27T05:20:00Z","@version":"1"}",
"tags": [
"request"
],
"@timestamp": "2017-01-27T05:20:00.000Z",
"route": "marketing#index",
"parameters": {},
"db": 6.11,
"status": 200
},
"fields": {
"@timestamp": [
1485494400000
]
}
}
My logstash.conf file looks like this
filter {
json {
source => "message"
}
mutate {
remove_field => [ "[_source][message][method]" ]
}
}
I have also tried
remove_field => [ "[method]"]
remove_field => [ "[message][method]"]
Also if it matters that data is coming in from redis using the input redis and codec => json.
Thanks in advance.