I'm using FileBeat to output to a file and then an ArcSight Regex file connector (usenonlockingwindowsfilereader=true, followexternalrotation=true ) pulls into ArcSight.
I have the rotation set to 2 because I don't really need to keep it.
However, the final file in the rotation becomes inaccessible to everyone, including Administrators. It seems to be corrupt.
Here is my output config:
This is what happens.
On first rotation, dnsjsonoutput.log is successfully renamed to dnsjsonoutput.log.1
On the next rotation, for some reason dnsjsonoutput.log.2 appears. It seems to have the oldest modified date, so I assume it's been renamed from dnsjsonoutput.log.1. However, the file is corrupt. I can't delete, take ownership, or even see the current owner of the file.
Eventually, I get the following error in the FileBeat log:
2016-12-14T15:15:12+11:00 CRIT Unable to write events to file: remove C:\dnslogs\dnsjsonoutput.log.2: Access is denied.
2016-12-14T15:15:12+11:00 INFO Error bulk publishing events: remove C:\dnslogs\dnsjsonoutput.log.2: Access is denied.
If I don't start the ArcSight Connector, dnsjsonoutput.log.2 is never created.
Any idea what might be happening here?
Edit: As A side note, I can manually delete dnsjsonoutput.log.1 before the next round of rotation happens, and there is no problem.