Filebeat source log file line length sanity check needed

[meritus]

Hi there.

This weekend i had a situation where my logfile got somehow corrupted having binary data at the beginning (0x00), and there normal txt data appended to it. And filebeat was scanning this file to put it into ES.

It turned out, that filebeat was using 20GB of RAM (RES) and tried to put it to ES. ES of course responded, that it's too large, but still filebeat used enormous amount of RAM.

Then, i started looking for a way to limit a log line size (for sanity reasons), and unfortunatelly did not found any way to limit this in filebeat.yml config file.

Am I missing something in configuration? If not, then perhaps it would be a good idea to introduce such a configurable parameter to avoid situation i faced. Not to mention, that crafted long line could lead to a server DOS because of filebeat eating more than available memory on that server ath the moment.

Best regards
Piotr Rybicki

[adrisr (Adrian Serrano)]

Hi,

Which version of filebeat are you using?

A few memory leaks have been fixed in recent versions.

[meritus]

Filebeat was at the latest version (6.4.2) at that moment

[kvch (Noémi Ványi)]

I think you are looking for the max_bytes option of the log input.

# Maximum number of bytes a single log event can have
# All bytes after max_bytes are discarded and not sent. The default is 10MB.
# This is especially useful for multiline log messages which can get large.
#max_bytes: 10485760

Example usage

filebeat.inputs:
- type: log
  enabled: true
  max_bytes: 1024

[meritus]

Sorry for a long delay.

It doesn't work this way for a zero-byte padded huge logfile (5GB). A very looong line at the beginning.

filebeat is latest: 6.4.3

revelant config:

- module: mongodb
  log:
    enabled: true
    var.paths: ["/var/log/mongodb/*.log"]
    input:
      max_bytes: "1024"
      fields.class: "log"

filebeat uses 5GB of RES, and then gives fatal error: out of memory

Perhaps this max_bytes parameter is checked against already loaded into memory line of logfile?

Best regards
Piotr Rybicki