The add_id processor in filebeat adds a unique value for [@metadata][_id]. You want duplicate events to have the same id, not a unique one. I would suggest a fingerprint filter, and configuring it to use a hash (not MAC) of whatever set of fields you think make an event unique.
Hello Badger, thanks for the response. Can you help me with this.
1d67c3b0 I, [2022-10-04T22:46:46.898239 #3722] INFO -- : Log-MESSAGE
After observing the logs the only field that is uniques I recognized is "2022-10-04T22:46:46.898239" with all 6 decimals.
How do I extract the that field to associate as a unique id. Here is what the documentation says on applying processors in Filebeat.