Changing file content during indexation

Elastic Stack Logstash
1

I have a logstah monitoring all files of type *.csv in a directory.
If i delete or change the content of one .csv (remove lines), will logstash be perturbated in its indexation task ?

2

Deleted files should be dropped the next time the filename patterns are scanned (every 15 seconds or whatever discovery_interval is set to). For updates it depends on how the files are updated. New file that's renamed into place or in-place update of existing file?

3

What would happen in these 2 cases :

  • renaming the file
  • updating in-place content of file (deleting lines or changing lines values)

..?

4
  • renaming the file

Not entirely sure.

  • updating in-place content of file (deleting lines or changing lines values)

Logstash will continue reading from the current position. If the update takes place afterwards in the file (i.e. a higher file offset) then it won't affect Logstash, but if it takes place earlier you'll see weird things. You can't delete a line without truncating the file and rewriting at least the remainder of the file, which might trigger Logstash's rotation logic (i.e. it thinks the file has shrunken and therefore has been rotated).

Just don't make in-place edits of files that Logstash monitors.

5

Thank you very much for your response, this confirms what i thought.
Logstash is not suitable in my study case.