Hi,
Currently monitoring a log file with Filebeat and sending content to Kafka. All is working as expected however, unless we raise the debug level do "debug" we have no idea if Filebeat worked properly.
We have a dissect processor configured which is working but unless the debug level is set for logging nothing shows up in the filebeat log file.
As for the Kafka output, with the debug level set for logging in Filebeat we can see "finished kafka batch" but nothing more than this.
As menitioned all is working properly; Filebeat is properly dissecting the log entries and sending the message in the proper format to our Kafka system. What's bothering us is that logging seems somewhat limited for troubleshooting purposes as "info" level gives us nothing and "debug" level, while showing us that the processing / dissecting has occured ( but nothing as for kafka output processing ) produces log files with "clutter" and unecessary information.
Is there some setting that can be applied so that, at the "info" level, we can log the activity of the filebeat processor and kafka output?
Any help or insight into this matter would be appreciated.
Thanks
José