What are the consequences of deleting the filebeat registry file? Will filebeat simply create a new blank registry file upon the next restart and reset its markers on all log files?
Running filebeat on Windows, I noticed that the shipper opened all of my older log files as well as my newer ones, resulting in a massive amount of active threads / CPU usage and backfilling my redis store.
After setting the 'ignore_older' field, I have configured filebeat to only ship my newest (<2hr) logs. However, the existing registry file continues to include open tabs on many of my older logs. These files remain open well past the 'close_older' setting as well (unsure as to why this is happening).
I want to clear this registry, and I don't care about shipping duplicate logs if it means my 'ignore_older=2h' can finally take effect so that filebeat won't hog the CPU and crash Redis.
I have referred here: Deleting Filebeat Registry File
- but not much of an answer is given to the original question apart from,
"registry-file is used to 'restart' from last known position. Deleting the complete registry file is not 'safe', as this might affect files currently being processed."
- Steffen Siering
Thank you,
Ravi