Filebeat High Memory Usage with multiline.negate=true

[derekcal]

I am running filebeat 6.2.3 and something I noticed is when I have a multiline pattern set and multiline.negate=true, I experience high memory usage.

Set multiline.negate=true 2:38 PM

11804 root 20 0 45.2g 4.4g 10m S 0.3 2.3 5513:48 filebeat

Set multiline.negate=false 2:42 PM

38722 root 20 0 329m 43m 9892 S 7.9 0.0 0:13.35 filebeat

Is there a reason setting multiline.negate=true would cause high memory usage?

[steffens (Steffen Siering)]

Memory usage depends on the number of files processed concurrently, queue size (default 4096 events), and the event sizes themselves. Multiline events are bigger by nature, so requiring more memory. Reason multiline.negate: true requires more memory is, because many more lines are captured into one event.
If your pattern is ill defined and captures the complete file by accident you might end up with very high memory usage.