# Recommendations for parsing 1000's ~10MB files to backfill elasticsearch

**URL:** <https://discuss.elastic.co/t/recommendations-for-parsing-1000s-10mb-files-to-backfill-elasticsearch/174114>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 27, 2019, 11:54am UTC](https://discuss.elastic.co/t/recommendations-for-parsing-1000s-10mb-files-to-backfill-elasticsearch/174114 "2019-03-27T11:54:18Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [March 29, 2019, 3:53pm UTC](https://discuss.elastic.co/t/recommendations-for-parsing-1000s-10mb-files-to-backfill-elasticsearch/174114/3 "2019-03-29T15:53:53Z")

</div>

What is the ratio between lines published and lines filtered out. The registry file keeps track of the file offset, but needs some IO to be written. If the ratio is somewhat 'bad', then the registry writes will slow down filebeat, as it also requires some fsync when writing the registry. Setting `filebeat.registry_flush: 1s` helps in this case (See [registry\_flush docs](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-general-options.html#_literal_registry_flush_literal)).

---

_[View the full topic](https://discuss.elastic.co/t/recommendations-for-parsing-1000s-10mb-files-to-backfill-elasticsearch/174114)._