FIlebeat from a Windows Network Share

in paths you might consider using \\winserver\logs\single*.log, such that after file rotation the rotated content will be send until file end (in case filebeat didn't process complete file yet).

After file rotation the new 'single.log' should have a new file id and new file should be picked up.

Is your application using some logging library, e.g. log4net? Is file really renamed or copied?

I don't see anything of interest in logs :confused: . Maybe run with "-v -e -d '*'" ? I'm basically missing a log line saying 'Force close file' (which is logged at INFO level). Can you grep the log file for this message? force_close_file option is active if file does not exist anymore or the volume/file id changed for same file name (file/volume ids uniquely identify a file).

As I don't have much experience with windows network shares on this low level i'd like to see what happens with volume and file id on file rotation.

If possible let's run some small experiment:

  1. check log directory is empty
  2. start filebeat it capture full log
  3. start application to generate log
  4. wait log file is rotated and filebeat stop processing
  5. stop filebeat
  6. copy registry file (default is .filebeat I think) into (registry1)
  7. delete registry file
  8. start filebeat and wait for another log rotate and stop filebeat
  9. copy registry file into (registry2)

What's the content of registry1 and registry2 after processing? The registry file stores file informartion like offset, path, volume/file id in json format. The volume/file id must be different for filebeat being able to detect a file was rotated.

If volume/file ids differ in registry files, but the 'Force close file' log message is still missing it points towards a bug in filebeat.

Sorry for the inconvenience, but properly supporting windows backed volumes is quite tricky in itself. Supporting network shared windows volumes is no easier. Any help is much appreciated.