Is there a way to prevent duplicates in a data stream ?
For a given index, specifying the _id gives us the guarantee that there will be no duplicate with same _id.
For data streams however, it does not work apparently.
We have a (homemade) data collector that has been launched 2 times. As our data stream rolled-over, the same data has been inserted in two different backing indices, so it is present two times in our data stream, which is a huge problem to us.
Are you aware of any solution to this problem ?