We are currently using filebeat 5.4.1 in our build environment The logs we are collecting are getting generated by the build software then removed after a build finishes. We are currently seeing a lot of access denied when the trying to delete those log files. We have experimented with several settings for filebeat, but nothing has helped so far. These logs are not huge usually less than 10 MB, the build script tries to delete the file, then waits 2 minutes, then tries again up til 5 times. (10 minutes)
This only happens intermittently, so it has been hard to reproduce across our 600+ Windows 10 build machines, but what we have seen is that filebeat reads the file, the index gets updated in the registry file, but the file is not "released". When I try to access the file filebeat is reading I get "access denied," even as an admin user I cannot see permissions/ownership of the file.
I cannot take ownership of the file neither by Administrator nor NT authority/System.
If I restart the filebeat service, or I kill the filebeat handle to the file, it gets deleted and the build process can move on.
Here are our current settings for a single prospector
close_renamed: True
close_removed: True
close_inactive: 30s
output.logstash:
# The Logstash hosts
hosts: [{{ filebeat_cfg.logstash.hosts }}]
bulk_max_size: 2048
loadbalance: true
worker: 4
Any ideas what could be causing this ?