Low level operations during _delete_by_query

[Rahul_Kumar3 (Rahul Kumar)]

What are the sequence of low-level operations that happens during the execution of _delete_by_query operation? On a high-level _delete_by_query operates as follows:

1. Multiple search requests are sequentially executed in order to find all the matching documents to delete.
2. Every time a batch of documents is found, a corresponding bulk request is executed to delete all these documents.

Basically, I wanted to understand the following things:

1. Whether the search request is routed to primary shard or happens at the node where the request is received?
2. Does the delete operation wait for all the matching documents to be found or immediately deletes the matched document?
3. Whether the delete operation is routed to primary shard or happens at the node where the request is received?
4. After the search operation is done how does delete operation know whether the version of the document it is trying to delete is the latest one?
5. Also, let's say that if somehow delete operation figures out that the version has been incremented after search operation then why does it throw version conflict. Shouldn't it just try to delete the latest version of the document?

[Christian_Dahlqvist]

I believe it runs the query as a scroll query, which runs in a context of the segments that were available when the request was made. Any updates that are performed after the delete-by-query was started will therefore not feature in the search input, which is why you get a version conflict when an updated document is found as the version is used during the delete phase to keep this potentially long-running process safe.

[Rahul_Kumar3 (Rahul Kumar)]

Hi @Christian_Dahlqvist,

@Christian_Dahlqvist

which is why you get a version conflict when an updated document is found as the version is used during the delete phase

So basically search operation searches in the context when the search request was started. I have two doubts here:

1. The search operation is performed on translog or on the Lucene index?
2. I am guessing that delete operation persists information about the deletion of a particular version of a document to the translog. So how does the delete operation know whether that particular version of the document is updated during the time when search operation finishes and delete operation starts? Is there some kind of intelligence build into translog that does not allow persisting information about an older version of a document?

[Christian_Dahlqvist]

@Rahul_Kumar3

The search operation is performed on translog or on the Lucene index?

The search only considers documents that have been committed to Lucene and are searchable.

@Rahul_Kumar3

I am guessing that delete operation persists information about the deletion of a particular version of a document to the translog. So how does the delete operation know whether that particular version of the document is updated during the time when search operation finishes and delete operation starts? Is there some kind of intelligence build into translog that does not allow persisting information about an older version of a document?

The delete takes documents in the translog into account. Get, set, update and delete operations on individual documents always consider all documents as it is not a search.