As is well-documented, Elastic provides "near real-time search capabilities". My data is typically ingested with through the Bulk API and we use a 30 second refresh interval as the data typically does not need to be available immediately. However, we do have a scenario where a user is presented with the data in the index and may want to "edit" it.
Our user experience allows the user to submit the change and wait while we re-ingest the document. We are setting the refresh=true param on the Bulk API request that processes the updated document. (we understand the impact of this setting and this is likely very low volume of these type of requests) We return back to the user after we get the response from the Bulk API request, and reload the page they are viewing. Our goal was to create a synchronous user experience where they would then see the change they had just made. Unfortuately, we are seeing that when we reload we do not see the change. It's within a second later that we can refresh and see the change. Our thought is that this may be due to the change getting ingested on the primary shard, however, it hasn't propogated out to the replicas so we are simply refreshing before the updated document exists everywhere.
The Refresh API sounds like it might be more robust, so instead of setting refresh=true on the Bulk API we could leave that to false and then call the refresh api in a subsequent request. Any recommendations on trying to achieve this type of reliable, synchronous user experience for document updates?
Thanks!