Provided Grok expressions do not match field value: Fortinet 7.12.1

Elastic Stack Beats
1

Hello,

recently I started a module for logs from fortigate firewalls.

- module: fortinet
  firewall:
    enabled: true

    # Set which input to use between tcp, udp (default) or file.
    #var.input: udp

    # The interface to listen to syslog traffic. Defaults to
    # localhost. Set to 0.0.0.0 to bind to all available interfaces.
    #var.syslog_host: localhost

    # The port to listen for syslog traffic. Defaults to 9004.
    #var.syslog_port: 9004
    var.input: "file"
    var.paths: ["/var/log/syslog/filename.log"]

But when I check for documents with data I can see that all documents contains error message

error.message
Provided Grok expressions do not match field value: [2021-04-28T09:02:37.250477+02:00 10.10.191.10 date=2021-04-28 time=09:02:37 devname=\"asd_asd_f60e_001_01\" devid=\"FGT60ETK18025222\" eventtime=1619593357182083292 tz=\"+0200\" logid=\"0000000013\" type=\"traffic\" subtype=\"forward\" level=\"notice\" vd=\"root\" srcip=10.10.10.12 srcport=50144 srcintf=\"v1824\" srcintfrole=\"lan\" dstip=10.10.12.17 dstport=8180 dstintf=\"v1823\" dstintfrole=\"lan\" srccountry=\"Reserved\" dstcountry=\"Reserved\" sessionid=3425247988 proto=6 action=\"close\" policyid=39 policytype=\"policy\" poluuid=\"7df5a1b0-f3ed-51e8-153e-76fd4c1aff1a\" policyname=\"application flow4\" service=\"TCP/8180\" trandisp=\"noop\" duration=1 sentbyte=216 rcvdbyte=112 sentpkt=4 rcvdpkt=2 appcat=\"unscanned\" srchwvendor=\"Samsung\" devtype=\"Phone\" srcfamily=\"Galaxy\" osname=\"Android\" srchwversion=\"A\" srcswversion=\"8.0.0\" mastersrcmac=\"00:15:5d:bf:21:0f\" srcmac=\"00:15:5d:af:83:0f\" srcserver=0 dsthwvendor=\"Oracle\" dstosname=\"Linux\" masterdstmac=\"00:21:f6:d0:cc:19\" dstmac=\"00:21:f6:a0:ce:19\" dstserver=0]

Is It a problem with format of fortigate logs?

2

What version of filebeat and elastic? It looks like your logs are not in the expected format. See https://www.github.com/elastic/beats/tree/master/x-pack%2Ffilebeat%2Fmodule%2Ffortinet%2Ffirewall%2Ftest%2Ffortinet.log for the format that the grok pattern is built for.

3

Both 7.12.1

5

It's because it's still expecting the syslog priority string, ex. <188>. See the link I posted above to the sample data that's used to test the module. How are u getting that different data in the front, we can always update the grok pattern but curious if this is an isolated incident or could be more widespread.

1 Like
6

For you specifically, you can update the grok pattern for the ingest pipeline for the fortinet firewall, should be named something like filebeat-7.12.1-fortinet-firewall-pipeline to something like below

%{TIMESTAMP_ISO8601} %{IPORHOST} %{GREEDYDATA:syslog5424_sd}$

That way it works for ur setup and the rest of the pipeline should process normally.

1 Like
7

It worked, thank you very much. I don't have enough information to point if this is an isolated incident or not.

8

Happy to help.

Are you using the default logging settings for your firewall? I see you are using the log file input, not the syslog input. Are you running Filbeat on the firewall or using another tool to put the logs into a file?

1 Like
9

Logs are sent via syslog to my linux and save to file. Filebeat reads from this file. I am only the receiver of logs I do not have view into fortigate configuration.

10

Ok, that makes sense. Then whatever program is saving the syslog to the file is probably adding that data to the front. If you're able you could use the syslog input on Filebeat and have the firewall logs sent directly to Filebeat without the need for another tool nor storing them to a file. That should resolve the issue of having to modify the ingest pipeline. If not, then you should be good with the current modifications. Just know that when you upgrade the ingest pipeline will be upgraded too and you may need to make the changes again.

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.