Drop if message contains text from a List of Strings

Elastic Stack Logstash
1

Hi,

I have a pre-defined set of Strings that i want to hard code in the conf file.

if "some text " in [message] or "some text" in [message] {
drop {}
}

I want to create an array of strings and loop through them accordingly, and whenever there is a string match i want to drop that particular event.

I have tried with the ruby scripts but for me it was not working.

2

Have a look at the regex option in the translate filter (bonus, you can load your blacklist from a file).
If you want the whole of the message field value to match the translate "key" then set regex => false.
Example:


input {
  generator {
    lines => [
      '222101333',
      '123456789',
      'abc103def',
      'xyz301mno'
    ]
    count => 1
  }
}

filter {
  translate {
    field       => "[message]"
    destination => "[matched]"
    dictionary  => [ "100", "drop",
                     "101", "drop",
                     "102", "drop",
                     "103", "drop" ]
    exact       => true
    regex       => true
  }
  if [matched] == "drop" {
    drop {}
  }
}

output {
  stdout {
    codec => rubydebug {metadata => true}
  }
}

Output:

{
      "sequence" => 0,
    "@timestamp" => 2018-08-07T16:24:00.311Z,
      "@version" => "1",
          "host" => "Elastics-MacBook-Pro.local",
       "message" => "123456789"
}
{
      "sequence" => 0,
    "@timestamp" => 2018-08-07T16:24:00.312Z,
      "@version" => "1",
          "host" => "Elastics-MacBook-Pro.local",
       "message" => "xyz301mno"
}