I have a very simple setup for Logstash 2.3.2 to process CSV files. But some of the files never get processed. This topic has been discussed elsewhere on this forum, but not solved that I can see. (Each CSV has a newline at the end of each file.)
I keep getting debug messages that contain the following line:
file grew: /Users/Eric/dev/data/test_long.csv: old size 0, new size 207272
If I copy a few lines out of the problem file and save them in a new file, they get processed successfully. Although, it is interesting to note that the debug message I see indicates that there is more to be processed in the file:
file grew: /Users/Eric/dev/data/test.csv: old size
4773, new size
5049
The logstash conf file I'm using is very simple:
input {
file {
path => "/Users/Eric/dev/data/*.csv"
start_position => "beginning"
ignore_older => 0
}
}
filter {
csv {
separator => ","
}
}
output {
stdout {
codec => rubydebug
}
}
Setup:
Mac Pro, OS X El Capitan , Version 10.11.6
Java SE Runtime Environment (build 1.8.0._31-b13)
Logstash version 2.3.2 (installed using 'brew install logstash')
Thanks in advance for your feedback.