# Always keep under docs.deleted after Delete By Query API

**URL:** <https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183>\
**Category:** Elasticsearch\
**Created:** [February 9, 2018, 7:34am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183 "2018-02-09T07:34:47Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [February 9, 2018, 7:34am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183/1 "2018-02-09T07:34:47Z")

</div>

Hi guys,

These days I deleted documents by Query API. Below is my command to delete my document by Query API.

```bash
curl -XPOST 'http://elasticsearch:9200/myindex/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "range" : {
        "createTime" : {
           "lte" : "2018-02-08"
        }
    }
  }
}
'

```

After took effect to delete document by Query API. But it always keep under `docs.deleted`. Here you are to see:

```bash
[user@elk ~]$ curl -XGET "http://elasticsearch:9200/_cat/indices/myindex ?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open myindex STGtht6tes2upE4WrSOKTg 5 1 46420892 3271000 29.3gb 14.3gb

```

In additional, it still occupied the `store.size`.

If anyone know how to release the deleted size. Restart all cluster node?

Regards/Eason

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 9, 2018, 7:56am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183/2 "2018-02-09T07:56:54Z")

</div>

A `_forcemerge` should help.

Note that is the reason using timebased indices is much better!

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [February 9, 2018, 8:09am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183/3 "2018-02-09T08:09:03Z")

</div>

HI @dadoonet,

You mean need to `_forcemerge` and will really delete it?  
If user timebased indices, it will delete whole indice, which is not my expectation. So that I use `_delete_by_query`.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 9, 2018, 8:34am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183/4 "2018-02-09T08:34:34Z")

</div>

Yes. This is what I meant. [Force merge API | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html)

> If user timebased indices, it will delete whole indice, which is not my expectation. So that I use \_delete\_by\_query.

But your request shows: `"lte" : "2018-02-08"` here?

Which is basically the same thing as doing something like:

```auto
DELETE 2017-*,2018-01-*,2018-02-01,2018-02-02,2018-02-03,2018-02-04,2018-02-05,2018-02-06,2018-02-07,2018-02-08

```

But my version of it is a way much more efficient than your way!

So if it's a one time only operation, using the DELETE BY QUERY could be fine as long as you remove only a small subset of the data. Otherwise, if you meant to keep like 10% of the data, it's better to use REINDEX API and drop the old index.  
If it's a task you are going to run every day, then use timebased indices and then use Curator to automate that index removal every day.

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [February 9, 2018, 8:55am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183/5 "2018-02-09T08:55:59Z")

</div>

Oh. One thing to be mentioned, I only have one index to store all data, not separate them by date postfix for indice name. So I only one way to delete the document of indice instead of indices.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 9, 2018, 9:27am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183/6 "2018-02-09T09:27:10Z")

</div>

That's exactly what I meant.  
It's wrong IMO and you should revisit your architecture.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 9, 2018, 9:27am UTC](https://discuss.elastic.co/t/always-keep-under-docs-deleted-after-delete-by-query-api/119183/7 "2018-03-09T09:27:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.