[SOLVED] Drop filter issue

Elastic Stack Logstash
1

Hi all,

I try to drop the field "message" in logstash:

filter {
      if [type] == "test" {
          grok {
              match => {"message" => "%{TIMESTAMP_ISO8601:syslog_host_time} %{IP:client} %{WORD:severity} %{DATESTAMP_IOS:client_date}\: \%%{FACILITY_WORD:facility}-%{POSINT:int_severity}-%{WORD:mnemonic}: %{GREEDYDATA:short_message}"}
               add_tag => [ "test" ]
               match => { "path" => "%{YEAR:log_year}" }
          }
          mutate {
              add_field => { "@source" => "%{client}" }
          }
          dns {
              nameserver => "127.0.0.1"
              reverse => [ "@source" ]
              action => "replace"
          }
          date {
              timezone => "Europe/Paris"
              match => [ "client_date", "YYYY MMM dd HH:mm:ss z", "YYYY MMM dd HH:mm:ss.SSS 'CET'", "YYYY MMM dd HH:mm:ss.SSS 'CEST'", "Y
          }
          drop {
            remove_field => [ "message" ]
          }
      }
  }

But my problem is when I try that I have no more logs.

Do you know why ?

But it works when I remove the drop filter and I try with mutate filter like:

mutate {
    add_field => { "@source" => "%{client}" }
    remove_field => [ "message" ]`
}

Thanks in advance for your answers,
Alex

2

The drop filter drops whole events, not individual fields of events. Replace drop with mutate.

3

Thanks for your quick answer !

The behavior looks like you said, but I'm abit confused when I read the documentation:

Drop filter

- remove_field

    . Value type is array
    . Default value is [] 

If this filter is successful, remove arbitrary fields from this event.

What is the purpose of remove a specific field if the whole event is drop ?

Have a good day,
Alex

4

Some options like add_field and remove_field apply to all filters even though they might not always make sense.

5

Thanks !

Have a good day,
Alex