# ES 1.5 Delete By Query API not working

**URL:** <https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899>\
**Category:** Elasticsearch\
**Created:** [December 1, 2017, 9:07am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899 "2017-12-01T09:07:49Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andrey\_Deineko](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Andrey\_Deineko](https://discuss.elastic.co/u/Andrey_Deineko)\
**Post date:** [December 1, 2017, 9:07am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/1 "2017-12-01T09:07:49Z")

</div>

Here is the original question asked on Stackoverflow: [elasticsearch - ES 1.5 Delete By Query API not working - Stack Overflow](https://stackoverflow.com/questions/47572129/es-1-5-delete-by-query-api-not-working)

I am using an old version on Elasticsearch - 1.5.

Problem: I need to delete a lot of documents, like few hundred thousands up to few millions. I have all the info about the records, including it's `_id`s - so array of `_id`s is what I want to use.

Scale problem: I had this deletion in the loop before, but ES is inconsistent when performing a lot of subsequent operations in a high speed. Thus I decided to look for a bulk delete.

I am trying to make use of [delete by query API](https://www.elastic.co/guide/en/elasticsearch/reference/1.5/docs-delete-by-query.html).

Docs states:

> ```
> curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
> "query" : {
> "term" : { "user" : "kimchy" }
> }
> }
> '
> 
> ```

What I'm doing:

```
curl -XDELETE 'http://localhost:9200/my_index/logs/_query' -d '{
  "query" : {
    "terms" : { "_id" : ["AVTD6fhLAn35BG25xbZz", "AVTD6fhLAn35BG25xbaC"] }
  }
}
'

```

The response is:

```
{
  "found":false,
  "_index":"my_index",
  "_type":"logs",
  "_id":"_query",
  "_version":1,
  "_shards":{"total":2, "successful":1, "failed":0}
}

```

It looks like ES is confused because it seems to be looking for a document with `_id` of `_query`... Very strange.

And it does not remove any of documents. How do I make it work and actually delete these records?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 9:20am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/2 "2017-12-01T09:20:18Z")

</div>

If you are deleting by id I would recommend usi8ng the bulk API instead. Issuing a delete by query operation for a small set of unique ids does not sound very efficient to me.

---

<div class="post-metadata">

**Author:** ![Andrey\_Deineko](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Andrey\_Deineko](https://discuss.elastic.co/u/Andrey_Deineko)\
**Post date:** [December 1, 2017, 9:34am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/3 "2017-12-01T09:34:20Z")

</div>

Thanks for the reply! I will look into bulk api, but the deleting by query looks the most natural to me - why do you see this approach inefficient?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 9:37am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/4 "2017-12-01T09:37:21Z")

</div>

If you wanted to delete based on the results of a query, it would be the natural choice. As you have the document ids, using the bulk API to explicitly delete multiple documents at a time is more natural in my opinion. This approach can also be parallelised and has minimal overhead.

---

<div class="post-metadata">

**Author:** ![Andrey\_Deineko](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Andrey\_Deineko](https://discuss.elastic.co/u/Andrey_Deineko)\
**Post date:** [December 1, 2017, 3:12pm UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/5 "2017-12-01T15:12:34Z")

</div>

I am trying the following:

```auto
bulk_delete = "{\":delete\":{\":_index\":\"my_index\",\":_type\":\"logs\",\":_id\":\"AVTD6fhLAn35BG25xbZz\"}}\n"
Typhoeus.post("http://elastic_host/my_index/logs/_bulk", body: bulk_delete)

```

And the response I get is as follows:

```auto
response.body
#=> {"found"=>false, "_index"=>"my_index", "_type"=>"logs", "_id"=>"_bulk", "_version"=>1}

```

```auto
response.response_headers

HTTP/1.1 404 Not Found
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=UTF-8
Content-Length: 108
Connection: keep-alive

```

```auto
response.code
#=> 404

```

I don't see where I go wrong about it.. Thank you very much for taking a look!

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 4, 2017, 8:06am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/6 "2017-12-04T08:06:03Z")

</div>

Why does it say `:delete`, `:_index`, `:_type` and `:_id` in your bulk\_delete string?

---

<div class="post-metadata">

**Author:** ![Andrey\_Deineko](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Andrey\_Deineko](https://discuss.elastic.co/u/Andrey_Deineko)\
**Post date:** [December 4, 2017, 8:29am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/7 "2017-12-04T08:29:50Z")

</div>

Should it not?

It is how I understood it from docs, - I have to provide a method (`delete`), index, type and id of the object to delete, no?

[https://www.elastic.co/guide/en/elasticsearch/reference/1.5/docs-bulk.html:](https://www.elastic.co/guide/en/elasticsearch/reference/1.5/docs-bulk.html:)

```auto
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 4, 2017, 8:31am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/8 "2017-12-04T08:31:45Z")

</div>

> [@Andrey\_Deineko](#):
>
> ":delete"

The example from the docs is correct, but your example looks wrong (see example above). You seem to have a few too many `:` in your string.

---

<div class="post-metadata">

**Author:** ![Andrey\_Deineko](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Andrey\_Deineko](https://discuss.elastic.co/u/Andrey_Deineko)\
**Post date:** [December 4, 2017, 8:49am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/9 "2017-12-04T08:49:18Z")

</div>

Oh, I see what you mean.

I tried the following now:

```auto
bulk_delete = "{ \"delete\" : { \"_index\" : \"my_index\", \"_type\" : \"logs\", \"_id\" : \"AV_v9fh4g0tG8Fm0jtzS\" } }"
puts bulk_delete
#=> { "delete" : { "_index" : "my_index", "_type" : "logs", "_id" : "AV_v9fh4g0tG8Fm0jtzS" } }
Typhoeus.post("http://#{AppConfig.elastic_host}/my_index/logs/_bulk", body: bulk_delete)

```

and the response code is `400` with the following info:

```auto
{"error":"ActionRequestValidationException[Validation Failed: 1: no requests added;]","status":400}

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 4, 2017, 8:50am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/10 "2017-12-04T08:50:31Z")

</div>

You need a newline after each bulk action, even the last one. Thy this:

```auto
curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "delete" : { "_index" : "my_index", "_type" : "logs", "_id" : "AV_v9fh4g0tG8Fm0jtzS" } }
'

```

---

<div class="post-metadata">

**Author:** ![Andrey\_Deineko](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Andrey\_Deineko](https://discuss.elastic.co/u/Andrey_Deineko)\
**Post date:** [December 4, 2017, 9:09am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/11 "2017-12-04T09:09:43Z")

</div>

Changed my payload to the following (added new line char to the end)

```auto
"{ \"delete\" : { \"_index\" : \"my_index\", \"_type\" : \"logs\", \"_id\" : \"AV_v9fh4g0tG8Fm0jtzS\" } }\n"

```

and worked like charm.

Thank you so much Christian for your involvement and help - very much appreciated!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 1, 2018, 9:09am UTC](https://discuss.elastic.co/t/es-1-5-delete-by-query-api-not-working/109899/12 "2018-01-01T09:09:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.