Logstash prune nested fields

Elastic Stack Logstash
1

HI TEAM,

I am using ELK version 7.0. Facing an issue with prune filter.
I need to blacklist exact nested fields coming from filebeat. When i tried with below format, prune is unable to blacklist host.architecture field.
Example:
prune {
blacklist_names => [ "^[host][architecture]$" ]
}

Regards,
GR.

2

Hi there,

I don't think prune is able to work properly with nested fields. As far as I remember there was a github issue about it.

Anyway, you could get that result with a ruby filter

filter {
  ruby {
    code => "
      def remove_fields(blacklist, event)
        blacklist.each { |field| event.remove(field) }
      end

      blacklist = [
        '[host][architecture]',
        '[whatever][else]'
      ]

      remove_fields(blacklist, event)
    "
  }
}
1 Like
3

Indeed. It is a documented feature

This filter currently only support operations on top-level fields, i.e. whitelisting and blacklisting of subfields based on name or value does not work.

2 Likes
4

Ok I remembered well, then. Thanks for the clarification.

Yet @gangireddy_l, can you achieve what you want using the ruby filter I wrote?

Obviously you could write it like following too

filter {
  ruby {
    code => "
      [
        '[host][architecture]',
        '[whatever][else]'
      ].each { |field| event.remove(field) }
    "
  }
}

but it is a bit less readable IMO.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.