How not to get double entries in my datastream using the Index API

Elastic Stack Elasticsearch
1

Hi,

We are using the Index API to post data to my datastream. I read from a queue and then perform a POST but I do have a timeout setting of 10 seconds. What sometimes happens it seems is that elastic takes more then 10 seconds so I throw a timeout on the connection and rollback to the queue. When I then retry I get a double entry as the first call probably went through but it took longer then 10 seconds. I thought I could change this perhaps by not using POST but using a PUT and supplying the doc id so I would create an UPSERT and prevent double entries but you can't update in a data stream. I can make the timeout setting higher but I don't want to go too high as this might cause stuck threads. Any ideas for this?

best regards,
Hugo

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.