Filebeat 7.7 panw module sends THREAT, but not TRAFFIC logs

Elastic Stack Beats
1

I am testing Filebeat 7.7 with the panw module, and am receiving THREAT type logs, but not TRAFFIC type logs. Elasticsearch version is also 7.7.

filebeat.yml config:

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
output.elasticsearch:
  hosts: ["host1", "host2"]
  loadbalance: true
  username: "user"
  password: "pass"
  protocol: https
  ssl.certificate_authorities: ["/path/ca.pem"]
  ssl.key: "/path/ssl.key"
  ssl.key_passphrase: 'passphrase'
  ssl.certificate: "/path/ssl.crt"

panw.yml config:

- module: panw
  panos:
    enabled: true
    var:
      input: "file"
      paths: ["/path/to/file.log"]

I'm using pretty basic config here without customization, the log file contains both threat and traffic logs. Threat logs ingest without issue, traffic logs never show up.

Any ideas where to look for logs that may point to the problem? filebeat and elasticsearch logs don't show errors that I can find.

2

Could you please share the debug logs Filebeat (./filebeat -e -d "*")?

3

@kvch Using the debug parameter you showed me, I was able to see immediately that it is being parsed incorrectly. There is an IP in the "message_subtype" field, which of course gets dropped as only the TRAFFIC or THREAT message_subtype's are processed in this module.

Here's a sample of the stdout with debug parameter:

Publish event: {
  "@timestamp": "2020-07-06T18:52:34.990Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.7.0",
    "pipeline": "filebeat-7.7.0-panw-panos-pipeline"
  },
  "observer": {
    "serial_number": "TRAFFIC"
  },
  "log": {
    "offset": 102246,
    "file": {
      "path": "/path/to/file.log"
    }
  },
  "tags": [
    "pa_test",
    "pan-os"
  ],
  "input": {
    "type": "log"
  },
  "host": {
    "name": "hostname.local.com"
  },
  "_temp_": {
    "message_subtype": "192.168.1.1",
    "generated_time": "0.0.0.0",
    "message_type": "drop"
  },
  "ecs": {
    "version": "1.5.0"
  },
  "agent": {
    "ephemeral_id": "8e98e9ea-f1e0-477d-a4f0-979ded422c11",
    "hostname": "hostname.local.com",
    "id": "240caa00-2e60-42e3-bf40-5b99ac213836",
    "version": "7.7.0",
    "type": "filebeat"
  },
  "message": "Jul  1 05:00:00 firewall 2020/07/01 05:00:00,0008C100362,TRAFFIC,drop,192.168.1.1,192.168.1.2,0.0.0.0,0.0.0.0,RuleName,,,not-applicable,vsys1,Zone1,Zone2,ethernet1/11,,sharedPanorama,0,1,59589,443,0,0,0x4000,tcp,deny,66,1,2020/07/01 05:00:00,0,any,66,0",
  "event": {
    "module": "panw",
    "dataset": "panw.panos",
    "timezone": "-04:00",
    "created": "0008C100362"
  },
  "fileset": {
    "name": "panos"
  },
  "service": {
    "type": "panw"
  }
}

Using this command:

GET /_ingest/pipeline/filebeat-7.7.0-panw-panos-pipeline

I can look at the pipeline, but I'm no expert here. It could be in the pipeline or the painless scripts being called. Wherever the field parsing happens.

Offhand guess, I'd say maybe it's the version of Palo-Alto logs? Or maybe we customized our log outputs so the number or order of fields is different than expected?

4

Filebeat modules only supported the default log format of these products. What version are you running?

I am looking into the problem to see if we can adjust the module to fit your case.

5

This message is formatted differently than the pipeline expects:

Jul  1 05:00:00 firewall 2020/07/01 05:00:00,0008C100362,TRAFFIC,drop,192.168.1.1,192.168.1.2,0.0.0.0,0.0.0.0,RuleName,,,not-applicable,vsys1,Zone1,Zone2,ethernet1/11,,sharedPanorama,0,1,59589,443,0,0,0x4000,tcp,deny,66,1,2020/07/01 05:00:00,0,any,66,0",

First it needs an extra field at the start, which in our samples is "1,", but even if that's fixed, other fields don't match what the parser expects, here's an example of a valid log:

Nov 30 16:09:08 PA-220 1,2018/11/30 16:09:07,012801096514,TRAFFIC,end,2049,2018/11/30 16:09:07,192.168.15.207,184.51.253.152,192.168.1.63,184.    51.253.152,new_outbound_from_trust,,,apple-maps,vsys1,trust,untrust,ethernet1/2,ethernet1/1,send_to_mac,2018/11/30 16:09:07,22751,1,55113,443,    16418,443,0x400053,tcp,allow,7734,1758,5976,36,2018/11/30 15:59:04,586,computer-and-internet-info,0,32091112,0x0,192.168.0.0-192.168.255.255,U    nited States,0,16,20,tcp-fin,0,0,0,0,,PA-220,from-policy,,,0,,0,,N/A,0,0,0,0

You'll see the initial "1" field after the syslog header, and then that there are two timestamp fields while the provided logs have only the first one.

This is the PAN-OS documentation we used to develop the module:

https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/monitoring/use-syslog-for-monitoring/syslog-field-descriptions/traffic-log-fields.html

We support all the versions documented in the above document, but none conforms to the provided log.

Which device / version are you using? Maybe there is some configuration option in the device to select different output formats?

6

Alternatively, if no option is found to adapt the logs to the documentation, you can try configuring CEF output as described in:

https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/monitoring/use-syslog-for-monitoring/syslog-field-descriptions/custom-logevent-format.html#id1dba4621-ff18-49f6-b613-50f678ce8351

and use Filebeat's cef module to ingest them.

7

Versions here range from 7.1 to 8.1 which is within compatible ranges according to filebeat docs.
I just confirmed that we customized our traffic output logs. Some fields had been trimmed from the output.
I've now tested a default output and they are ingesting but with the wrong time zone in kibana, which appears to be a known issue where there is no timezone defined in the log itself:

I'm a little uncertain how to fix this. I tried some of the fixes such as replacing the module/panw/panos/ingest/pipeline.yml in the pull request, as well as upgrading from filebeat 7.7.0 to 7.7.1 and it's still not the correct timezone in Kibana.

Is there something I can provide to help diagnose the issue?

8

I was able to fix the timezone by using the drop_fields processor as recommended here:
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-panw.html#_time_zone_support_11
I thought this was referring to the filebeat panw ingest pipeline processors at first, but the drop_fields processor config goes in the filebeat.yml. My config ended up looking like this:

processors:
  - drop_fields:
      fields: ["event.timezone"]
      ignore_missing: true

Thanks for the help folks.

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.