Hi there,
I am testing the rollover feature to address our big indice issue. I can see the issue of double indexing.
Basically, our app sends bulk request to cluster. The app would retry the bulk request if any errors (failed or partially failed). We use guid as doc id to make sure our write is upsert.
As in this scenario, the retry will happen in either network latency when receiving ack or partial failed bulk. If we use rollover, there is a highly chance the original write went to old indice and the retry would go to new rolled indice. We can't afford duplicate documents on searching. What are the solutions for this issue?
Thank you!