Blacklisting with Ruby Filter

[pkward]

Hi,

I'm looking to blacklist 1000+ nested fields. I've read in another forum that it's not possible to use the prune filter for nested filters, but I can with a ruby filter. I wanted to know is there a way to blacklist with a ruby filter using a wildcard based on the first object field? e.g., if I had a field called "File.Type.Extension", could I create a ruby filter that could blacklist all fields that begins with File or essentially File*? Or could you provide me guidance based

"""

filter {
  ruby {
    code => "
      def remove_fields(blacklist, event)
        blacklist.each { |field| event.remove(field) }
      end

      blacklist = [
        '[File]*'
      ]

      remove_fields(blacklist, event)
    "
  }
}

"""

[leandrojmp (Leandro Pereira)]

The example you gave can easily be done with just the prune filter.

It works with top-level fields, in this case the File json object, if you want to blacklist everything starting with File.*, then you can use the prune filter.

If this is not the case you will need to write a ruby code that will remove the fields according to what you do not want.

[pkward]

Ok, I wasn't sure because I saw elsewhere that I couldn't prune nested fields. I will attempt before coming here next time.