I have my datascience logs on a linux machine and I use Filebeat installed on the client linux machine which forwards logs to Logstash, we parse it and forward those logs to Elasticsearch. The issue is that whenever there is a reboot happening on the linux machine, we receive duplicate logs at elasticsearch. I have tried to overcome the duplicate logs issue at logstash but I am curious why this duplicate logs appear? also is their solution to this issue which can be done at filebeat level?
All your feedbacks are appreciated.