I want to prevent duplicate documents from being inserted into new elastic index, using watcher actions. I am not sure whether it is possible or not.
Here's the scenario.
Watchers, scheduler is set to 15 seconds and input range time set as last 15 min.
Condition. set to specific use cases & rules. whenever those use cases conditions occur an alert will be triggered.
So, After condition met I'm pushing those records into newly elastic index using watcher action.
Suppose In watchers where i'm taking input from another elastic index where logs are coming continuously real time. I have set scheduler 15 second and last 15 min time range. so after the based on certain use case condition if, i push those records in another index using watcher action. at this point i have following queries.
Is there any possibility of duplication when i pushing those specific records in new elasticsearch index ?
If yes, then how can prevent from duplication of same event ?