# Filebeat rc1 resends full data set data upon random restart (registry file not updated properly)?

**URL:** <https://discuss.elastic.co/t/filebeat-rc1-resends-full-data-set-data-upon-random-restart-registry-file-not-updated-properly/63580>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 21, 2016, 8:00am UTC](https://discuss.elastic.co/t/filebeat-rc1-resends-full-data-set-data-upon-random-restart-registry-file-not-updated-properly/63580 "2016-10-21T08:00:09Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![shog](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shog/32/12138_2.png) [@shog](https://discuss.elastic.co/u/shog)\
**Post date:** [October 24, 2016, 3:35pm UTC](https://discuss.elastic.co/t/filebeat-rc1-resends-full-data-set-data-upon-random-restart-registry-file-not-updated-properly/63580/5 "2016-10-24T15:35:58Z")

</div>

Hi @ruflin

- My registry files are actually quite huge as each filebeat operates on 2k new files created every day. The biggest registry file I see on rc1 cluster is almost 5MB big and it has a state fo 20k files inside.
- Because of other issue I've encountered in [Filebeat beta1 resends random data upon every restart (registry file not updated properly?) - #7 by shog](https://discuss.elastic.co/t/filebeat-beta1-resends-random-data-upon-every-restart-registry-file-not-updated-properly/61813/7) I'm only using clean\_inactive: 36h, with clean\_removed: false,
- As you can guess from my registry size I have tons of remarks of old files that are long gone, but filebeat is still complaining about them with "$file should have been dropped, but couldn't as state is not finished.". Is there any way to enforce cleanup of those?
- I restart filebeat two times daily after my file rotation script finish and I do have shutdown\_timeout: set to 60 already. Filebeat never requires that long to stop.
- I will try to prepare data and share it over email.

> [@ruflin](#):
>
> We also found an issue in the migration of old states: [Fix state loading in case a state falls under ignore\_older after restart by ruflin · Pull Request #2830 · elastic/beats · GitHub](https://github.com/elastic/beats/pull/2830) But this is more related to not clean states from the registry file.

I believe I have a nodes where I fully nuked registry file and above issue occured again.

---

_[View the full topic](https://discuss.elastic.co/t/filebeat-rc1-resends-full-data-set-data-upon-random-restart-registry-file-not-updated-properly/63580)._