Duplication in Filebeat to Elasticsearch data pushing

Setting IDs or Unique IDs are not yet supported. Deduplication via IDs is something we will definitely want to support in the future.

The duplicate events are normally a sign of I/O errors while ingesting data. Have you checked your logs for I/O errors? Maybe it's a matter of queue/bulk size for processing events in Elasticsearch.