Logstash multiline codec plugin won't keep blank lines

Elastic Stack Logstash
1

Hi,

I am working on the correct settings for the multiline codec plugin. I want to join all lines between two \r\n into a single event and it works well, except that blank lines are dropped.

I'm running Logstash v7.16.1 with the following debug setup:

Logstash config:

input {
    file {
        path => "${SKY_IMPORT_PATH}"
        file_completed_action => "log"
        file_completed_log_path => "/dev/null"
        mode => "read"
        exit_after_read => true
        file_chunk_size => 100000000000
        sincedb_path => "/dev/null"
        codec => multiline {
            pattern => "\r$"
            negate => true
            what => "next"
        }
    }
 }

output {  
  stdout {}
}

Test file:

header\r\n
line1:a\n
b\n
\n
c\n
d\r\n

If I run logstash with multiline codec disabled, it outputs each line as a separate event, as expected, i.e. separate events with the messages header\r, line1:a, b, <empty string>, c, d\r respectively. Note that the blank line is not dropped.

If I run logstash with the multiline codec enabled, it joins the lines as intended except for dropping the blank line.
I get two events with messages header\r and line1:a\nb\nc\nd\r but I would expect (and need) the second message to be line1:a\nb\n\nc\nd\r, i.e. containing the blank line from the test file.
Also, If I add more line breaks into my test file, every group of consecutive \n is collapsed into a single \n.

Can somebody help me saving my blank line, please? :slight_smile:

Thanks,
Samuel

2

A multiline codec drops blank lines. See Guy's comments on this issue and this post in the forum.

3

Thanks for the quick response.