Registry cleanup is not implemented it. We are still discussion the options. Follow this issue for more details: https://github.com/elastic/beats/issues/1600
About the files like /flow_base_dir/dump/flows-201606150045.dump. So not change happened to these files? Because the above means, that offset > file.Size() so it assumes the file was truncated.
There are some minor changes in alpha4 related to the offset, but should not be related to the above issues. But worth a try 
What I ended up doing is to update the bash script that generates these dump files, to rotate them between 12 files with 5 min increments. So then I don't have to worry about disk cleanup. In this case, I do see the file truncate in the logs, but this time it's for overwriting the file. So I think from that perspective it's good.
Every ten minutes I will see this in the logs, is this normal?
2016-06-16T15:00:17Z WARN client/metadata fetching metadata for all topics from broker broker-2.service.consul:7000
2016-06-16T15:00:17Z WARN client/metadata fetching metadata for all topics from broker broker-0.service..consul:7000
Good to hear you found a solution. I hope to fix this in the long term from the filebeat side.
About the warning message: This seems to come from the Kafka library (sarama) we use. TBH it sounds reasonable to me, not sure why it goes into WARN. @steffens knows perhaps more.
sarama lib only provides log.Println (used for INFO and ERROR messages). WARN level is used as 'compromise'. The message about fetching metadata is very normal.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.