[Please excuse my poor English...]
If output the file to the same file with a line codec of the file output, the data will be lost.
my logstash.conf
input {
beats {
port => "5044"
}
}
filter {
grok{
match => {
"message" => "^%{INT:num}$"
}
}
if "_grokparsefailure" in [tags] {
drop { }
}
mutate {
convert => {"num" => "integer"}
}
}
output {
if [num] == 1111111111 {
file {
path => "test/test.txt"
codec => line {
format => 'test1="%{num}",test2="%{num}"'
}
}
}
else if [num] == 2222222222 {
file {
path => "test/test.txt"
codec => line {
format => 'test1="%{num}",test2="%{num}"'
}
}
}
}
A file with only 1111111111 or 222222222 was used as input.
Outputs to the same input file differ.
1st - 110,992 KB
2nd - 110,945 KB
Just by comparing the two, there is a difference in capacity by 47KB.
When the output file is divided.
logstash.conf
input {
beats {
port => "5044"
}
}
filter {
grok{
match => {
"message" => "^%{INT:num}$"
}
}
if "_grokparsefailure" in [tags] {
drop { }
}
mutate {
convert => {"num" => "integer"}
}
}
output {
if [num] == 1111111111 {
file {
path => "test/test1.txt"
codec => line {
format => 'test1="%{num}",test2="%{num}"'
}
}
}
else if [num] == 2222222222 {
file {
path => "test/test2.txt"
codec => line {
format => 'test1="%{num}",test2="%{num}"'
}
}
}
}
1st - 55,665KB, 55,665KB
2nd - 55,665KB, 55,665KB
We can see that there is no difference in capacity between the two attempts.
The conclusion is that if you print the same file with line codec, the data will be lost.
Is it a bug?