Logstash reading events from March 14th on March 16th

Elastic Stack Logstash
10

Thanks team.

I read about the deduplication issue.

I Want to remove the duplicate events inside Logstash filter how could I do that? I mention the events below please have a look and suggest.

Duplicate events are overwritten.
If you have ILM policy where you can write to index for 2 days and if logstash will try to write that index ( duplciate events are updated not dropped)

we need to find a way to drop the event if it is overwriting or index based on logstash timestamp no issue will be seen, if it is from timestamp generated from the event metadata then issue starts.