Hi all. Is there a way to reduce the size of filebeat logs? I deployed filebeat to a Docker host, everything works as expected, all container logs are sent to Elasticsearch. However, after a few days, json log file in /var/lib/docker/containers was smth like 20GB. It was filebeat log file, other logs files were much smaller. Is it expected? Do I need to somehow monitor filebeat docker log?
I am not sure the root of your problem is Filebeat unless you have modified the default logging configuration. By default the maximum size of a log file (produced by Filebeat) is 10 MB. If the size exceeds it, the file gets rotated. The number of log files created by Filebeat is 7. Thus, total size of the logs should not be bigger than 70 MB.
You can configure logging of Filebeat in a few ways to avoid the issue you are having at the moment.
- Decrease logging level
You can set the logging level to warning or error, so the number of log messages are decreased. It can be adjusted using the optionlogging.level: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html#level - Decrease the size of the log files or the number of rotated files
Set a smaller size for the log by configuringlogging.files.rotateeverybytes: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html#_logging_files_rotateeverybytes Or you can lower the number of log files by settinglogging.files.keepfiles: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-logging.html#_logging_files_permissions
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.