No data from Filebeat 7.7 - Using Palo Alto module

[hueyg]

Have a working already establish 7.6 cluster processing multiple beats from multiple hosts. Attempting to setup another beat to process Palo Alto file logs, but unlike the others I am using the built in PaloAlto module to ship the logs.

I am using the 7.7 version of Filebeat, the Palo Alto module is confirmed enabled, and it appears to be process the specified log, but no data is showing up in my Elastic cluster. There is a new index that must be related because it follows the filebeat index template naming convention but there are zero documents in the index.

Here is section of the filebeat log:

`2020-05-20T18:22:45.095-0500    DEBUG   [harvester]     log/log.go:107  End of file reached: /var/log/hosts/pci-pa-2/pci-pa-2.log; Backoff now.

2020-05-20T18:22:45.101-0500 DEBUG [elasticsearch] elasticsearch/client.go:217 PublishEvents: 9 events have been published to elasticsearch in 8
.031577ms.
2020-05-20T18:22:45.101-0500 DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [152: 0, 9]
2020-05-20T18:22:45.101-0500 DEBUG [publisher] memqueue/eventloop.go:535 broker ACK events: count=9, start-seq=1977, end-seq=1985

2020-05-20T18:22:45.101-0500 DEBUG [publisher] memqueue/ackloop.go:128 ackloop: return ack to broker loop:9
2020-05-20T18:22:45.101-0500 DEBUG [publisher] memqueue/ackloop.go:131 ackloop: done send ack
2020-05-20T18:22:45.101-0500 DEBUG [acker] beater/acker.go:64 stateful ack {"count": 9}
2020-05-20T18:22:45.101-0500 DEBUG [registrar] registrar/registrar.go:356 Processing 9 events
2020-05-20T18:22:45.101-0500 DEBUG [registrar] registrar/registrar.go:326 Registrar state updates processed. Count: 9
2020-05-20T18:22:45.101-0500 DEBUG [registrar] registrar/registrar.go:411 Write registry file: /var/lib/filebeat/registry/filebeat/data.jso
n (1)
2020-05-20T18:22:45.102-0500 DEBUG [registrar] registrar/registrar.go:404 Registry file updated. 1 states written.
2020-05-20T18:22:46.096-0500 DEBUG [processors] processing/processors.go:112 Fail to apply processor client`

That "Fail to apply process client" repeats for every event that it reads from the Palo Alto file I have specified. So it appears it is reading the file, but the pipeline is having an issue with the data and possibly not shipping it out?

[hueyg]

Found the solution in my case. Turns out the team who handles the PaloAlto had modified the format of the original log files so they were being presented to filebeat in a non standard format. They were no longer in a CSV format.

I am assuming this is what caused the panw module not to be able to correctly parse and map the fields. Once the team set the log back to the original out of the box format the panw module worked like a champ.