# Filebeat Fortinet have error message

**URL:** <https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 10, 2020, 11:20am UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261 "2020-12-10T11:20:52Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [December 10, 2020, 11:20am UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/1 "2020-12-10T11:20:52Z")

</div>

Hi @P1llus,

I saw you're the person that give more comment on Filebeat Fortinet module, so I directly ask for help.

I just send my fortinet log into my rsyslog server and save it into the file then I enabled the fortinet modules in Filebeat. I already followed all the procedures to enable the module in [this URL](https://github.com/elastic/beats/issues/19010) .

I have error.message as follow:

```auto

    "message": "Dec 10 17:59:54 192.100.254.68 date=2020-12-10 time=17:59:54 devname=\"FW_VPN_Jasindo_DRC\" devid=\"FG3H1E5819900316\" logid=\"0000000013\" type=\"traffic\" subtype=\"forward\" level=\"notice\" vd=\"root\" eventtime=1607597994 srcip=192.168.32.38 srcname=\"VCENTER-DRC-BSD\" srcport=46418 srcintf=\"port2\" srcintfrole=\"lan\" dstip=192.168.203.6 dstport=445 dstintf=\"port1\" dstintfrole=\"wan\" poluuid=\"b67b21c8-9185-51e9-efbb-a32dfe6a402c\" sessionid=2387369696 proto=6 action=\"server-rst\" policyid=3 policytype=\"policy\" service=\"SMB\" dstcountry=\"Reserved\" srccountry=\"Reserved\" trandisp=\"noop\" duration=1 sentbyte=104 rcvdbyte=80 sentpkt=2 rcvdpkt=2 appcat=\"unscanned\" devtype=\"Windows PC\" devcategory=\"Windows Device\" osname=\"Windows 7 / 2008 R2\" unauthuser=\"vcenter-drc-bsd$\" unauthusersource=\"kerberos\" mastersrcmac=\"00:50:56:ae:1a:90\" srcmac=\"00:50:56:ae:1a:90\" srcserver=1",
    "error": {
      "message": "Provided Grok expressions do not match field value: [Dec 10 17:59:54 192.100.254.68 date=2020-12-10 time=17:59:54 devname=\\\"FW_VPN_Jasindo_DRC\\\" devid=\\\"FG3H1E5819900316\\\" logid=\\\"0000000013\\\" type=\\\"traffic\\\" subtype=\\\"forward\\\" level=\\\"notice\\\" vd=\\\"root\\\" eventtime=1607597994 srcip=192.168.32.38 srcname=\\\"VCENTER-DRC-BSD\\\" srcport=46418 srcintf=\\\"port2\\\" srcintfrole=\\\"lan\\\" dstip=192.168.203.6 dstport=445 dstintf=\\\"port1\\\" dstintfrole=\\\"wan\\\" poluuid=\\\"b67b21c8-9185-51e9-efbb-a32dfe6a402c\\\" sessionid=2387369696 proto=6 action=\\\"server-rst\\\" policyid=3 policytype=\\\"policy\\\" service=\\\"SMB\\\" dstcountry=\\\"Reserved\\\" srccountry=\\\"Reserved\\\" trandisp=\\\"noop\\\" duration=1 sentbyte=104 rcvdbyte=80 sentpkt=2 rcvdpkt=2 appcat=\\\"unscanned\\\" devtype=\\\"Windows PC\\\" devcategory=\\\"Windows Device\\\" osname=\\\"Windows 7 / 2008 R2\\\" unauthuser=\\\"vcenter-drc-bsd$\\\" unauthusersource=\\\"kerberos\\\" mastersrcmac=\\\"00:50:56:ae:1a:90\\\" srcmac=\\\"00:50:56:ae:1a:90\\\" srcserver=1]"
    }

```

Just to check the Grok in the Fortinet pipeline:

```auto
{
    "grok": {
      "patterns": [
        "%{SYSLOG5424PRI}%{GREEDYDATA:syslog5424_sd}$"
      ],
      "field": "message"
    }
  }

```

The fortinet config as follow:

```auto
config log syslogd setting
      set status enable
      set server "rsyslog"
      set mode legacy-reliable
      set port 5514
      set facility local7
      set source-ip ''
      set format defaults
end

```

Also this is my fortinet.yml

```auto
# Module: fortinet
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.10/filebeat-module-fortinet.html

- module: fortinet
  firewall:
    enabled: true

    # Set which input to use between tcp, udp (default) or file.
    var.input: file

    # The interface to listen to syslog traffic. Defaults to
    # localhost. Set to 0.0.0.0 to bind to all available interfaces.
    #var.syslog_host: 0.0.0.0

    # The port to listen for syslog traffic. Defaults to 9004.
    #var.syslog_port: 9004

    # Set paths for the log files when file input is used.
    var.paths: ["/datalog/192.100.254.68/*.log"]

    # Toggle output of non-ECS fields (default true).
    var.rsa_fields: false

    # Set custom timezone offset.
    # "local" (default) for system timezone.
    # "+02:00" for GMT+02:00
    var.tz_offset: +07:00

  clientendpoint:
    enabled: false

    # Set which input to use between udp (default), tcp or file.
    var.input: file
    # var.syslog_host: localhost
    # var.syslog_port: 9510

    # Set paths for the log files when file input is used.
    # var.paths: 

    # Toggle output of non-ECS fields (default true).
    # var.rsa_fields: true

    # Set custom timezone offset.
    # "local" (default) for system timezone.
    # "+02:00" for GMT+02:00
    # var.tz_offset: +07:00

  fortimail:
    enabled: false

    # Set which input to use between udp (default), tcp or file.
    var.input: file
    # var.syslog_host: localhost
    # var.syslog_port: 9529

    # Set paths for the log files when file input is used.
    # var.paths:
  
    # Toggle output of non-ECS fields (default true).
    # var.rsa_fields: true

    # Set custom timezone offset.
    # "local" (default) for system timezone.
    # "+02:00" for GMT+02:00
    # var.tz_offset: +07:00

  fortimanager:
    enabled: false

    # Set which input to use between udp (default), tcp or file.
    #var.input: file
    # var.syslog_host: localhost
    # var.syslog_port: 9530

    # Set paths for the log files when file input is used.
    # var.paths:
 
    # Toggle output of non-ECS fields (default true).
    # var.rsa_fields: true

    # Set custom timezone offset.
    # "local" (default) for system timezone.
    # "+02:00" for GMT+02:00
    # var.tz_offset: +07:00

```

Is there any workaround to make the fortinet index working?

BTW, I just enable panw.yml and it works flawlesly.  
Appreciate your help.

[Edit]  
Elasticsearch: 7.10.0  
Filebeat: 7.10.0

Regards,  
Fadjar340

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [December 11, 2020, 4:20am UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/2 "2020-12-11T04:20:07Z")

</div>

After do some research, finaly I found the problem.

In the grok processor for log file input, I modified and added this:

```auto
%{SYSLOGTIMESTAMP} %{SYSLOGHOST} %{GREEDYDATA:syslog5424_sd}$

```

The original is:

```auto
%{SYSLOG5424PRI}%{GREEDYDATA:syslog5424_sd}$

```

Perhaps you need some pipe into the grok processor if the input using file as follow:

```auto
%{SYSLOG5424PRI}%{GREEDYDATA:syslog5424_sd}$|%{SYSLOGTIMESTAMP} %{SYSLOGHOST} %{GREEDYDATA:syslog5424_sd}$

```

This grok statement will handle direct input udp/tcp and file as well.

Now, my fortinet logs working as expected.

Regards,  
Fadjar340

---

<div class="post-metadata">

**Author:** ![Sergii\_Patsora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sergii_patsora/32/81258_2.png) [@Sergii\_Patsora](https://discuss.elastic.co/u/Sergii_Patsora)\
**Post date:** [December 22, 2020, 12:35pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/3 "2020-12-22T12:35:19Z")

</div>

Hello,

I have similar problem with ELK 7.10 and FortiOS v6.0.11 - none of these pipelines work for me.

> Provided Grok expressions do not match field value: [Dec 22 14:15:18 fg200d date=2020-12-22 time=14:15:17 devname="FG200D4Q26914264" devid="FG200D4Q26914264" logid="1059028704" type="utm" subtype="app-ctrl" eventtype="app-ctrl-all" level="information" vd="root" eventtime=1608639317 appid=16009 srcip=10.10.2.232 dstip=77.75.149.201 srcport=53234 dstport=80 srcintf="port1" srcintfrole="lan" dstintf="port2" dstintfrole="undefined" proto=6 service="HTTP" direction="outgoing" policyid=1 sessionid=380781585 applist="appcontrol\_MTU" appcat="Update" app="MS.Windows.Update" action="pass" hostname="2.tlu.dl.delivery.mp.microsoft.com" incidentserialno=1278502712 url="/filestreamingservice/files/3b79c97b-09ea-41fb-92f1-4589f4e2f3c3?P1=1608639938&P2=402&P3=2&P4=OmtIZ1o%2bo5%2bKUyN9lEFm0SjQ4QJ5ebb76H%2fqxg2cJ7iOnOnTCDLJq4yu5i9r%2f9J8yXdoYwZNvQ63qSxQfVqg4A%3d%3d" msg="Update: MS.Windows.Update," apprisk="elevated"]

Can anyone give me any suggestion? Thanks.

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [December 22, 2020, 12:57pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/4 "2020-12-22T12:57:14Z")

</div>

I have document that explain how to change the ingest pipeline of the fortinet.  
In my case, I use log file as input, not from UDP/TCP port and all the information was ingested.  
If you interested, please message me..

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [December 22, 2020, 2:03pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/5 "2020-12-22T14:03:38Z")

</div>

Open the following ingest pipeline for fortinet under your filebeat

```auto
${FILEBEAT_HOME}/module/fortinet/firewall/ingest/pipeline.yml

```

And add the new pattern

```auto
    - '%{SYSLOGTIMESTAMP}\s*%{SYSLOGHOST}\s*{GREEDYDATA:syslog5424_sd}$'

```

Final config should looks like

```auto
description: Pipeline for parsing fortinet firewall logs
processors:
- set:
    field: event.ingested
    value: '{{_ingest.timestamp}}'
- grok:
    field: message
    patterns:
    - '%{SYSLOGTIMESTAMP}\s*%{SYSLOGHOST}\s*{GREEDYDATA:syslog5424_sd}$'
    - '%{SYSLOG5424PRI}%{GREEDYDATA:syslog5424_sd}$'
- kv:
    field: syslog5424_sd
    field_split: " (?=[a-z\\_\\-]+=)"
    value_split: "="
    prefix: "fortinet.tmp."
    ignore_missing: true
    ignore_failure: false
    trim_value: "\""

```

---

<div class="post-metadata">

**Author:** ![fadjar340](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fadjar340/32/43610_2.png) [@fadjar340](https://discuss.elastic.co/u/fadjar340)\
**Post date:** [December 22, 2020, 2:12pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/6 "2020-12-22T14:12:00Z")

</div>

@ylasri, thank you for your snippet.  
Some condition also related to the Date when the syslog using timezone not UTC.  
In my case, my timezone +07:00, the pipeline save next 7 hours of my localtime.

Below my procedure to change the ingestion:

- In `Kibana` - `Stack Management`, do some changes of `Ingest Node Pipelines` - `filebeat-7.10.0-fortinet-firewall-pipeline`
- Edit `filebeat-7.10.0-fortinet-firewall-pipeline`
- Find `Grok` in the second line below `Set`, upper `Key-value (KV)`
- Select the `Grok`, `Manage` and `Edit`
- Find `Patterns`
- Change the patterns as follow:

```auto
%{SYSLOG5424PRI}%{GREEDYDATA:syslog5424_sd}$|(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource}+(?: %{SYSLOGPROG}:|) %{GREEDYDATA:syslog5424_sd}$

```

- Update the `Grok` processor
- Find 2 `DATE` processor below `SET` upper `GSUB`
- In the second `DATE` processor where: `Condition (optional)` as follow: `ctx.fortinet?.firewall?.tz == null`, fill the `Timezone (optional)` with your timezone, ie: `+07:00`, as time zone of the syslog server. This value will **Prevent the timestamp not following your timezone but Next 7 hours (in my case)**
- Update the `Date` Processor
- Save Pipeline at the bottom of the pipeline page

The Filebeat ingestion pipeline also have error not only Fortinet, also F5 and Checkpoint in my case using log file input using rsyslog. In my experience, Paloalto works flawlessly...

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [December 22, 2020, 2:20pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/7 "2020-12-22T14:20:26Z")

</div>

I added a new [FR](https://github.com/elastic/beats/issues/23246) to update the module  
You can comment with differents example if you wish

---

<div class="post-metadata">

**Author:** ![Sergii\_Patsora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sergii_patsora/32/81258_2.png) [@Sergii\_Patsora](https://discuss.elastic.co/u/Sergii_Patsora)\
**Post date:** [December 22, 2020, 2:34pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/8 "2020-12-22T14:34:04Z")

</div>

Appreciate your help - now everything works as it should. Thank you all!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 19, 2021, 4:34pm UTC](https://discuss.elastic.co/t/filebeat-fortinet-have-error-message/258261/9 "2021-01-19T16:34:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.