Applying conditionals to grok filter

[vinod_hy]

Hi All,

I have log in the below format,
2017-06-08 19:05:56.222 loglevel=DEBUG,sysId=467,package=com.MyPackage1,class=MyClass.java,method="sendMessage",Id=679,message=Hello First Message.

Below is my filter,
filter{
grok
{
match => {"message" =>"%{TIMESTAMP_ISO8601:timestamp} %{GREEDYDATA:keyval}"}
}
kv
{
source => "keyval"
field_split => ","
remove_field => [ "keyval"]
}
}

Now my requirement is to send the logs to elasticsearch whose package name matches to com.MyPackage1. Rest all messages belonging to other packages needs to be ignored.
For that i made the below changes,
filter
{
grok
{
match => {"message" =>"%{TIMESTAMP_ISO8601:timestamp} %{GREEDYDATA:keyval}"}
}
kv
{
source => "keyval"
field_split => ","
remove_field => [ "keyval"]
}
}
filter{
if "com.MyPackage1" not in [package]
{
drop { }
}
}

This solution works fine. Now my another use case is that i should be able to drop certain fields present in the logs. Say suppose i want to drop method or class. I can add those fields in remove_field section of the kv filter as below,

remove_field => [ "keyval","method","class"]
It takes care of it.

But if i add the package field in remove__field as below,
remove_field => [ "keyval","package"]

then i wont to able to perfrom drop as drop is dependent of package filter to drop the messages.

How to solve this situation. Is there a better way. Please help me

[vinod_hy]

Hi all,
This is what i did to solve the issue,
filter{
grok
{
match => {"message" =>"%{TIMESTAMP_ISO8601:timestamp} %{GREEDYDATA:keyval}"}
}
kv
{
source => "keyval"
field_split => ","
}
}
filter{
if "com.MyPackage1" not in [package]
{
drop { }
}
else
{
kv
{
source => "keyval"
remove_field => [ "keyval","message","method" ]
}
}

}

[vinod_hy]

In the first kv filter, i am just spliting the keyval. In the second kv filter, i am removing the unwanted fields.
Please let me know is there a better solution.