Filebeat beta1 resends random data upon every restart (registry file not updated properly?)

Your assumptions are correct.

However I see something very interesting now. I'm on a box now where registry file wasn't updated for several hours.

[root@host1 filebeat]# date
Mon Oct  3 16:21:38 UTC 2016

[root@host1 filebeat]# ls -l /var/lib/filebeat/registry
-rw-r----- 1 filebeat filebeat 1223255 Oct  3 02:04 /var/lib/filebeat/registry

And it also looks like the very last entry having either "regist" or "state" was also at this time.

[root@host1 filebeat]# grep states /var/log/filebeat/filebeat.log | tail -n 1
2016-10-03T02:04:53Z INFO Non-zero metrics in the last 30s: libbeat.publisher.published_events=2086 libbeat.logstash.published_and_acked_events=2293 libbeat.logstash.call_count.PublishEvents=10 libbeat.logstash.publish.read_bytes=378 libbeat.logstash.publish.write_bytes=1211622 libbeat.publisher.messages_in_worker_queues=9 registrar.writes=2 registrar.states.update=4274

There is nothing related to registry/states afterwards.

And I'd like to stress out that there is plenty of other events in filebeat log (like harvester ones) and Filebeat operates completely fine and it's also delivering events all the time without any issue.

[root@host1 filebeat]# ps -eo comm,lstart | grep filebeat
filebeat-god    Mon Oct  3 01:58:22 2016
filebeat        Mon Oct  3 01:58:22 2016

So the time was at follow
01:58 filebeat started
02:04 registry file was updated for last time
02:04 onwards :: delivering events normally

In other words on-disc registry file was not updated for 14hrs.

Restarting triggered entire data set to be rescanned. While stopping filebeat claimed that registry file was updated to current state. Unfortunately I did not copy it, so can check if it was updated compared to proper values. I believe I'll be able to reproduce this, so I'll post more details.

Judging from what I see in ES, all events after 02:04 were reimported again.