I have a CSV file which I imported with Logstash into Elasticsearch. This CSV is "growing" when time goes by. But when this happens Logstash completely imports the file again, this in stead of the last added records. I am using Windows and tried to solve this with sincedb_path to a csv.db file but it seems that this is not being used. Any suggestions would be great.
Exactly how does the file grow? Logstash can only track updates that are done by appending to original file.
There will be records added at the end of the file manually.
And what does "manually" mean?
People are adding records at the end of the spreadsheet on a regular basis.
Right, so the program rewrites the file from scratch in a new file and renames the new file over the old file. Logstash can't track such changes.
Ok, thank you. Then we have to find another solution for this.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.