Duplicate items in elasticsearch

Elastic Stack Logstash
1

Hello,

I'm using logstash forwarder to ship logs from centralized location to logstash servers and then to elasticsearch.

Recently, I've found that I have the same documents but with different ids twice in my Kibana. its basically the exact event, indexed twice. Тhis is happening for all document types.

What are the most common reasons for this?

I know this could be coming from different sources, but what I can't imagine so far is how to recreate this issue. Any help would be appreciated.

2

Is logstash-forwarder for some reason rereading the same old files?

3

I am not sure, but I've noticed something - 2016/01/27 09:35:17.510424 registry rotate: rename of .logstash-forwarder.new to .logstash-forwarder - rename .logstash-forwarder.new .logstash-forwarder: no such file or directory