[Solved]Mutate not working

[wakame (Tin Sheen)]

I upgraded logstash from 2.3.4 to 5.6.7, after the upgrade, mutate failed to remove any of the field. What do I need to check to make sure I have the right plugin?

LS 5.6.7
ES 2.4.1

[magnusbaeck (Magnus Bäck)]

I'm sure you have the right plugin. Perhaps you were relying on undocumented behavior that happened to change in recent Logstash releases. If you show us your configuration and an example input event we can help out.

[wakame (Tin Sheen)]

input {
kafka {group_id=>"xxxxxx" topics =>"xxxxx" type=>"route" bootstrap_servers=>"IP:9092" consumer_threads=>3} }
filter {
if [type] == "route" { mutate { remove_field => ["api_id"] } json{source=>"message" }} }

{
"_index": "xxxxxx",
"_type": "route",
"_id": "xxxxxxxxx",
"_score": null,
"_source": {
"api_id": "xxxxxxxx",
}

even just removing one field, I couldn't get it to work.

[wakame (Tin Sheen)]

I had to move

json{source=>"message" }

to the front of my filter and it started working.

[magnusbaeck (Magnus Bäck)]

The document clearly doesn't have a type field so the if [type] == "route" conditional is false.