Logstash file input W/ multiline cuts text

[urirosenberg]

Hi,
We are using logstash for ingesting files from a file system.
The problem we started to observe is that the input/content is being cut (so LS is indexing 2 events for certain files).

Here's a sample of our input config:
file {
path => "/dropzone/app/myfiiles/"
exclude => [".lz4", "data/", ".csv", ".tbl"]
start_position => "beginning"
ignore_older => 0
max_open_files => 1000
sincedb_path => "/dev/null"
close_older => 2
add_field => { "type" => "mytype"}
add_field => { "[@metadata][event_type]" => "mytype"}
codec => multiline {
pattern => "/../gm"
negate => true
what => previous
max_lines => 3000000
max_bytes => "120MiB"
auto_flush_interval => 5
charset => "ASCII"
}

more info:

1. The files being split/cut differ each day (not the same files).
2. The files DON'T contain special characters nor does LS outputs any errors in the logs
3. we are running multiple LS as docker containers (each with it's own pipeline)
4. Using LS 5.4