Filebeat not detecting all changes made to csv File

[Christian_Dahlqvist]

@ganessen23

As checked, once the script is launched again, it would normally create and overwrite the old file, hence the new lines would not necessarily be at the end of the file, and therefore tha'ts why Filebeat has not detected the new lines inserted.

This might explain why you are getting partial lines as well. If a line earlier in the file all of a sudden increases in size, it is possible a partial line could be read as Filebeat uses the file offset to keep track of what has been processed.

@ganessen23

Before I would modify the script to insert all «new» lines at the very end of the file, is there any way for Filebeat to re-analyse all lines once again as like «for loop», and add only the new lines?

Filebeat is not designed to handle this scenario, so I do not believe that is possible.

@ganessen23

Does copy paste and overwriting a file to the Filebeat Directory when Filebeat is always working with the file, may entails issue like that?

I think the altering of prior content may be at fault here rather than the method with which the file is copied.