ES DeleteByQuery fails to delete records sometimes

Elastic Stack Elasticsearch
1

Hi, We are using ES 2.0, on a 3 node cluster, with deleteByQuery plugin. The query sometimes fails to delete some or all of the documents. This behavior is not consistent. Sometimes, all docs are deleted, sometimes few and sometimes none. We also retry 3 times, if expected number of docs are not deleted. Below is the response of deleteByQuery API, but it does not have much information. There is no information related to this in ES logs. Please let us know what more things can we try.

Response: {
"_indices" : {
"_all" : {
"deleted" : 2140,
"failed" : 71,
"found" : 2211,
"missing" : 0
},
"cfileindex" : {
"deleted" : 2140,
"failed" : 71,
"found" : 2211,
"missing" : 0
}
},
"failures" : [],
"timed_out" : false,
"took" : 491
}

2

Hi, Can someone please take a look at this.

3

do you use java Client?
if you ,please post your code here

4

No, we are using our own http client written in cpp. This sends the http request as mentioned in docs.
https://www.elastic.co/guide/en/elasticsearch/plugins/2.0/delete-by-query-usage.html

5

Are you by any chance continuously updating documents in the index through some process?

6

We are doing insertion (not update) and deletion in a very short interval, like 5-10 secs, but we are not doing both at the same time. Can that result in partial deletion of documents. If yes, what is the correct way to delete all the desired records?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.