Filebeat System Module - Multi-line Log Generating One Log Per Line

I'm enabling the system module of filebeat for the first time on a Ubuntu server. I can see the logs but it's taking what looks like it's taking multiline logs and placing them into single logs per line. Some logs only have a single }. Everything is default during my setup except for locations of logstash.

Hi!

Maybe you need to override input settings and set properly the multiline settings: https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html

That didn't fix the issue. When setting that to true, I didn't see any logs come in.

The description of the system module says under the hood it will make sure multiline messages get treated as a single event but that doesn't appear to be happening.

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-system.html

Hmmm,

could you provide some logs samples please? I could try to reproduce your issue.

All the logs are just single lines so I can't really send them to you but here is a screenshot showing just the message field and you can see the end of one log and beginning of another but some individual logs are just closing } brackets. It only does this when I enable the system module.

Hi!

I see you are trying to parse Metricbeat's logs using system module :thinking:?
In principle I don't think this will work. As documentation mentions The system module collects and parses logs created by the system logging service of common Unix/Linux based distributions. .

So I figured out what was going on. Metricbeat by default will send it's logs to syslog even if you have it defined in the metricbeat.yml config file not to so the filebeat system module was still picking this up. I was able to override the default setting based on this post and this resolved the issue.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.