If you only want to keep one copy of an event where certain fields are the same you could use a fingerprint filter to hash those fields and use the result as the document_id in elasticsearch. The document will keep getting overwritten with the most recent event. See here for an example, although I would not recommend using SHA1 for anything these days.