I am planning to set a timeout for my bulk requests.
Is there a possible way to know, when the timeout exceeds, whether the request is still being processed by ES?
I want to retry some time after the timeout exceeds and I am afraid of a situation where I will send the same bulk again while it exists in the DB (but took a long time to insert in my first attempt)
If you want to index that data anyway, why would you set a timeout?
If you are having slow bulk requests, why not try to resolve that rather than introducing a timeout that could lead to duplicates getting created?
I don't have slow bulks. My usecase is:
An internal component sends bulk requests and waits for an answer. If there is no answer after 4 seconds, it retries.
I cant let elastic try processing a bulk forever...
What do you suggest in this case?
What is your average bulk latency? How frequently do you see latencies exceeding the 4 second timeout?
The system is in a development stage. There is no problem with bulk latency. We just want to define a max limit for waiting. We want to understand whether we can verify that there is no more processing for this request in ES server when we retry the request.