Logstash - How to Multiline correct

[matze1992]

I would like to analyse the following log. For this I have created a Grok-Filter in Logstash.

My Input-Filter looks like this:

file
    {
            path => "/etc/logstash/files.d/libview*"
            codec => multiline {
                    pattern => "^\/"
                    negate => true
                    what => previous
            }
            add_field => {"maschine" => "libview"}
            start_position => "beginning"
    }

This yields the following log:

{
Tape library space for drives: 500-51F
Path                Size     Active      Scratch / Qty       Free    Filesystem
----------------    ----   ---------   ----------------   ---------  ----------
/path      393.7G      0   0%      0   0%      0  361.4G 91%  /dev/sda2
/path/to
                    7.9T    3.4T 42%      1T 12%    488    3.5T 44%  name-alias0:/path/to (Repl)
/path/to
                    7.9T    3.5T 44%    1.2T 15%    584    3.1T 39%  name-alias2:/path/to (Repl)
/path/to
                    7.9T    3.5T 43%    1.2T 14%    588    3.3T 41%  name-alias1:/path/to (Repl)
/path/to
                    7.9T    3.7T 46%    1.3T 15%    593      3T 37%  name-alias3:/path/to (Repl)
/path/to
                    7.9T    3.7T 47%    1.2T 15%    598    2.9T 37%  name-alias2:/path/to (Repl)
/path/to
                    7.9T    3.7T 47%    1.1T 14%    571      3T 38%  name-alias0:/path/to (Repl)
/path/to
                    7.9T    3.5T 44%      1T 13%    523    3.3T 42%  name-alias3:/path/to (Repl)
/path/to
                    7.9T    3.8T 48%    1.2T 15%    590    2.9T 36%  name-alias1:/path/to (Repl)
/path/to
                    7.9T    3.5T 44%    1.2T 14%    570    3.2T 41%  name-alias0:/path/to (Repl)
/path/to
                    7.9T    3.4T 42%    1.1T 13%    550    3.4T 43%  name-alias2:/path/to (Repl)
/path/to
                    7.9T    3.6T 45%    1.1T 14%    543    3.2T 40%  name-alias1:/path/to (Repl)
/path/to
                    7.9T    3.6T 45%    1.2T 15%    597    3.1T 38%  name-alias3:/path/to (Repl)
/path/to
                    7.9T    3.5T 44%    1.2T 15%    598    3.2T 40%  name-alias2:/path/to (Repl)
                 ==================================================
         Totals:  102.4T   46.3T 45%     15T 14%   7393   41.1T 40%
----------------------------------------------------------------------
}

My problem now:
There is only one event logged in Elasticsearch.

Any idea?

I need a log like

/Path Size Active Scratch Qty Free Filesystem:/

without new lines.

[hartfordfive (Al)]

What if you change the codec this?:

codec => multiline {
   pattern => "^Tape library space for drives:"
   negate => true
   what => next
}

I haven't tested in Logstash with the file input although I've tested the equivalent config in Filebeat and it seems to work in the sense that it will at least return that whole event as a single one. If ever your interested, I've created a tool to help testing multi-line expressions in Filebeat configs although keep in mind it's in the early stages so some bugs may be encountered.