Elasticsearch delete_by_query 409 version conflict

Hi @Rahul_Kumar3,

I think the missing piece to make this safe is a refresh.

Delete by query basically does a search for the objects to delete and then deletes them with version conflict checking. Without a _refresh in between, the search done by _delete_by_query might return the old version of the document, leading to a version conflict when the delete is attempted.

To be certain that delete by query sees all operations done, refresh should be called, see: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html . Or you can use the refresh parameter on the previous indexing request, see: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-refresh.html

Notice that refreshing is not free. The last link above explains some of the trade-offs involved including the impact on indexing and search performance.

The default refresh interval is 1s, see: https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#dynamic-index-settings

2 Likes