Deleting the registry file

(rlonberg) #1

What are the consequences of deleting the filebeat registry file? Will filebeat simply create a new blank registry file upon the next restart and reset its markers on all log files?

Running filebeat on Windows, I noticed that the shipper opened all of my older log files as well as my newer ones, resulting in a massive amount of active threads / CPU usage and backfilling my redis store.

After setting the 'ignore_older' field, I have configured filebeat to only ship my newest (<2hr) logs. However, the existing registry file continues to include open tabs on many of my older logs. These files remain open well past the 'close_older' setting as well (unsure as to why this is happening).

I want to clear this registry, and I don't care about shipping duplicate logs if it means my 'ignore_older=2h' can finally take effect so that filebeat won't hog the CPU and crash Redis.

I have referred here: Deleting Filebeat Registry File

  • but not much of an answer is given to the original question apart from,

"registry-file is used to 'restart' from last known position. Deleting the complete registry file is not 'safe', as this might affect files currently being processed."
- Steffen Siering

Thank you,

(ruflin) #2

Stopping filebeat, deleting the registry and the starting filebeat again will create a new blank registry.

In filebeat 5.0 you can use the clean_* options to make sure your registry file does not grow over time.

Which version are you currently using? Some of the issues you mention above are pointing to one of the 1.x release where we had some issues with open files.

(system) #3

This topic was automatically closed after 21 days. New replies are no longer allowed.