Filebeat locking files (Access Denied)

We are currently using filebeat 5.4.1 in our build environment The logs we are collecting are getting generated by the build software then removed after a build finishes. We are currently seeing a lot of access denied when the trying to delete those log files. We have experimented with several settings for filebeat, but nothing has helped so far. These logs are not huge usually less than 10 MB, the build script tries to delete the file, then waits 2 minutes, then tries again up til 5 times. (10 minutes)

This only happens intermittently, so it has been hard to reproduce across our 600+ Windows 10 build machines, but what we have seen is that filebeat reads the file, the index gets updated in the registry file, but the file is not "released". When I try to access the file filebeat is reading I get "access denied," even as an admin user I cannot see permissions/ownership of the file.
I cannot take ownership of the file neither by Administrator nor NT authority/System.
If I restart the filebeat service, or I kill the filebeat handle to the file, it gets deleted and the build process can move on.

Here are our current settings for a single prospector

  close_renamed: True
  close_removed: True
  close_inactive: 30s

output.logstash:
  # The Logstash hosts
  hosts: [{{ filebeat_cfg.logstash.hosts }}]
  bulk_max_size: 2048
  loadbalance: true
  worker: 4

Any ideas what could be causing this ?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.