Drop incoming messages

[rosboc]

Hi,
i am using logstash 6.1.1 with following configuration and need to drop messages containing "type= traffic" string and i am not able to get it working.

input {
  
  udp {
       port => 2514
       type => "syslog"
       codec => plain { charset => "ISO-8859-1" }
       tags => ["fg_syslog"]
 }

}

filter{

    #ruby { code => "puts event.get('message')" }

    grok {
    match => [ "message", "type=traffic"]
    add_tag => ["drop"]
   }

}

output {

 #stdout { codec => rubydebug }

 if ( ('fg_syslog' in [tags]) and !('drop' in [tags]) ) {

    # send to logstash_fortigate
       syslog{
               host => "xxxxxx"
               port => "2514"
               protocol => "udp"
       }

    # send to QRadar
       fortigate_syslog_forwarder {
               host => "xxxxx"
               port => "514"
      }

  }
}

These are type of log logs logstash is receiving in input ( you can notice small differences) :

<181>Aug 2 10:05:39 df18.fcr01.dal09 date=2021-08-02 time=15:08:42 devname=df18-fcr01-dal09 devid=FG300C3913607629 logid=0000000013 type=traffic subtype=forward level=notice vd=firewall001 srcip=xxxxxxx srcport=36554 srcintf="v418-outside" dstip=xxxxxx dstport=10384 dstintf="v853-inside" poluuid=3555337c-1a66-51e9-0209-d11c3ea9350f sessionid=869404929 proto=6 action=deny policyid=65 dstcountry="United States" srccountry="United States" trandisp=noop service="tcp/10384" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=30 craction=131072 crlevel=high

<190>date=2021-08-02 time=15:08:29 devname="dft04-pod01-wdc04" devid="FG15DTT918800110" logid="0100032001" type="event" subtype="system" level="information" vd="firewall001" eventtime=1627916909142565217 tz="+0000" logdesc="Admin login successful" sn="1627916909" user="rboccia" ui="ssh(169.55.26.68)" method="ssh" srcip=xxxxxx dstip=xxxxxx action="login" status="success" reason="none" profile="VDOM Administrator" msg="Administrator rboccia logged in successfully from ssh(xxxxxxx)”

<189>date=2021-08-02 time=15:08:14 devname="dft01-pod01-dal09" devid="FG15DT3I16800226" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="firewall001" eventtime=1627916895028549746 tz="+0000" srcip=xxxxx srcport=36249 srcintf="v862-f-outside" srcintfrole="lan" dstip=xxxxxxx dstport=37037 dstintf="v852-f-inside" dstintfrole="lan" sessionid=91798369 proto=6 action="deny" policyid=0 policytype="policy" service="tcp/37037" dstcountry="United States" srccountry="China" trandisp="noop" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=30 craction=131072 crlevel="high"

Any idea how to solve this problem ?
Thanks.
Rosboc

[Badger]

The last two messages have quotes around the word traffic, so the grok will not match. If you want to make the quotes optional you could use

if [message] =~ /type="?traffic"?/ { drop {} }

[rosboc]

Hi,
do i have to use only your instructions in the filter section and remove other ones also from output section ?
Thanks

[Badger]

That is correct.

[rosboc]

It worked

is this correct ?

filter{
#ruby { code => "puts event.get('message')" }
if [message] =~ /type="?traffic"?/ { drop {} }
if [message] =~ /perf-stats/ { drop {} }
}

[Badger]

That looks OK to me.

[rosboc]

Perfect.
Thank you very much