No, it's not the usual problem. Using 6.2.4 I have a file that contains
1
1
2
2
3
They really are blank lines.
# od -a bar.txt
0000000 1 nl 1 nl nl 2 nl 2 nl nl 3 nl
With a stdin input, a multiline codec chops it up just fine. Running
/usr/share/logstash/bin/logstash -e '
input {
stdin {
codec => multiline { pattern => "^$" negate => true what => "previous" }
}
}
output { stdout { codec => rubydebug } }' < bar.txt
Gets me
{
"host" => "[...]",
"@version" => "1",
"tags" => [
[0] "multiline"
],
"@timestamp" => 2018-05-22T20:54:01.262Z,
"message" => "\n2\n2"
}
{
"host" => "[...]",
"@version" => "1",
"tags" => [
[0] "multiline"
],
"@timestamp" => 2018-05-22T20:54:01.260Z,
"message" => "1\n1"
}
However using a file input
/usr/share/logstash/bin/logstash -e '
input {
file {
path => "/path/to/bar.txt"
codec => multiline { pattern => "^$" negate => true what => "previous" }
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
output { stdout { codec => rubydebug } }'
Gets me nothing until I SIGINT logstash, at which point it flushes the whole file, with the blank lines deleted.
{
"@version" => "1",
"path" => "/path/to/bar.txt",
"@timestamp" => 2018-05-22T20:56:23.094Z,
"tags" => [
[0] "multiline"
],
"message" => "1\n1\n2\n2\n3",
"host" => "[...]"
}
auto_flush_interval does not fix anything, just saves me having to do the SIGINT. What am I missing?