Error Events with ">"

[akrog79]

Hello people!

I have a trouble with the ingestion of a anomaly events in fortigate.

The raw event is:

<185>logver=702032456 timestamp=1676305141 devname="FG200-E" devid="FG200ETK189243" vd="root" date=2023-02-13 time=17:19:01 eventtime=1676305141602895749 tz="+0100" logid="0720018433" type="utm" subtype="gorkapablo" eventtype="anomaly" level="alert" severity="critical" srcip=63.222.61.134 srccountry="France" dstip=109.111.111.203 dstcountry="Spain" srcintf="port13" srcintfrole="wan" sessionid=0 action="detected" proto=1 service="PING" count=236 attack="icmp_flood" icmpid="0x3721" icmptype="0x08" icmpcode="0x00" attackid=16777316 policyid=1 policytype="DoS-policy" ref="[http://www.fortinet.com/ids/VID16777316"](http://www.fortinet.com/ids/VID16777316%22) msg="anomaly: icmp_flood, 251 > threshold 250, repeats 236 times since last log, pps 30 of prior second" crscore=50 craction=4096 crlevel="critical"

As you can see, the field msg (anomaly: icmp_flood, 251 > threshold 250, repeats 236 times since last log, pps 30 of prior second) have a ">" symbol. This symbol makes me crazy because when appears, the event doesn't appear in Elastic, but if I erase it, the event appears.

My logstash conf.d file is:

input {
  tcp {
    port => 1025
    tags => ["fortigate"]
  }
}

filter {
    if "fortigate" in [tags] {
        grok {
                match => {"message" => "<(?<ruleID>.*)>(?<msg>.*)"}
        }
        kv { source => "msg" }
        mutate {
            rename => ["msg","message"]
            rename => ["type","log_type"]
            rename => [ "dst", "DestinationIP" ]
            rename => [ "dstip", "DestinationIP" ]
            rename => [ "dstport", "DestinationPort" ]
            rename => [ "dstintf", "DestinationZone" ]
            rename => [ "devname", "DeviceName" ]
            rename => [ "status", "Action" ]
            rename => [ "src", "SourceIP" ]
            rename => [ "srcip", "SourceIP" ]
            rename => [ "zone", "SourceZone" ]
            rename => [ "srcintf", "SourceZone" ]
            rename => [ "srcport", "SourcePort" ]
            rename => [ "service", "Application" ]
            rename => [ "policyname", "RuleName" ]
            rename => [ "action", "Action" ]
            rename => [ "rcvdbyte", "BytesReceived" ]
            rename => [ "sentbyte", "BytesSent" ]
            convert => {"DestinationZone" => "string"}
        }
}
}

output {
  if "fortigate" in [tags] {
          elasticsearch {
              hosts => ["X.X.X.X:9200"]
              index => "fortigate-%{+YYYY.MM.dd}"
              document_type => "fortigate"
              template => "/etc/logstash/elastic-fortigate-template.json"
              template_name => "fortigate"
              template_overwrite => true
          }
         }
 }

Someone can help me?

[leandrojmp (Leandro Pereira)]

The issue is that the character > breaks your grok filter as the first regex will capture everything between a < and a >.

In your case you do not even need to use grok as the fortigate messages will always have the same format, you can use the dissect filter to parse the message.

Replace your grok filter with this dissect and it will work:

    dissect {
        mapping => {
            "message" => "<%{ruleId}>%{msg}"
        }
    }