Logstash multiline codec plugin won't keep blank lines

Hi,

I am working on the correct settings for the multiline codec plugin. I want to join all lines between two \r\n into a single event and it works well, except that blank lines are dropped.

I'm running Logstash v7.16.1 with the following debug setup:

Logstash config:

input {
    file {
        path => "${SKY_IMPORT_PATH}"
        file_completed_action => "log"
        file_completed_log_path => "/dev/null"
        mode => "read"
        exit_after_read => true
        file_chunk_size => 100000000000
        sincedb_path => "/dev/null"
        codec => multiline {
            pattern => "\r$"
            negate => true
            what => "next"
        }
    }
 }

output {  
  stdout {}
}

Test file:

header\r\n
line1:a\n
b\n
\n
c\n
d\r\n

If I run logstash with multiline codec disabled, it outputs each line as a separate event, as expected, i.e. separate events with the messages header\r, line1:a, b, <empty string>, c, d\r respectively. Note that the blank line is not dropped.

If I run logstash with the multiline codec enabled, it joins the lines as intended except for dropping the blank line.
I get two events with messages header\r and line1:a\nb\nc\nd\r but I would expect (and need) the second message to be line1:a\nb\n\nc\nd\r, i.e. containing the blank line from the test file.
Also, If I add more line breaks into my test file, every group of consecutive \n is collapsed into a single \n.

Can somebody help me saving my blank line, please? :slight_smile:

Thanks,
Samuel

A multiline codec drops blank lines. See Guy's comments on this issue and this post in the forum.

Thanks for the quick response.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.