Hi @ruflin
- My registry files are actually quite huge as each filebeat operates on 2k new files created every day. The biggest registry file I see on rc1 cluster is almost 5MB big and it has a state fo 20k files inside.
- Because of other issue I've encountered in Filebeat beta1 resends random data upon every restart (registry file not updated properly?) - #7 by shog I'm only using clean_inactive: 36h, with clean_removed: false,
- As you can guess from my registry size I have tons of remarks of old files that are long gone, but filebeat is still complaining about them with "$file should have been dropped, but couldn't as state is not finished.". Is there any way to enforce cleanup of those?
- I restart filebeat two times daily after my file rotation script finish and I do have shutdown_timeout: set to 60 already. Filebeat never requires that long to stop.
- I will try to prepare data and share it over email.
I believe I have a nodes where I fully nuked registry file and above issue occured again.