Logstash: filter a dynamic field in the log line

In the below input, we need to filter out “COUNT_XXX123=1,SUCCESS_XXX123=1” even if we don’t have “sellerId=XXX123” field available.



filter {
        kv {
                field_split => ","
        mutate {
                remove_field => [  "COUNT_%{WORD}", "SUCCESS_%{sellerId}"]

In above case it is only filtering SUCCESS_XXX123 but not COUNT_XXX123.

remove_field => [  "COUNT_%{WORD}", "SUCCESS_%{sellerId}"]

This needs to be:

 remove_field => [  "COUNT_%{sellerId}", "SUCCESS_%{sellerId}"]

You might also find the prune filter useful.

Thanks for you reply, I wanted to ask if we don't have "sellerId" field available can we just remove all fields like "COUNT_*"?

Did you look at the prune filter as I suggested?

Yes, prune filter with blacklist_names worked for this, Thanks a lot for your help!

 prune {
                    blacklist_names => ["SUCCESS_[A-Za-z0-9]*", "COUNT_[A-Za-z0-9]*" ]