Throttle Filter Logstash

[Bigboy0706]

Hi
As there are some deduplicated log ingressing into logstash
That's why I would like to use the Throttle Filter to filter out the log
The config is listed as below

filter {
     if [name] == "pack_incident_response_process_events" {
        throttle {
          before_count => -1
          after_count => 1
          period => 10
          max_age => 20
          key => "%{hostIdentifier_s}%{columns_cmdline_s}%{columns_path_s}"
          add_tag => "throttledd"
        }
  }

But for the result, only one event is able to shown-up without the throttledd tags which mean the other different log had been filtered-out if I drop the tags = throttledd, but i just want to filter out the same cmdline log.

Please advise.

Thanks

[Badger]

The throttle filter adds a tag to indicate that events have the same [key] value. What to do with that information is left to you. For example, you can

if "throttled" in [tags] { drop {} }

to discard the duplicates if that is what you want.