# Delete by query limitation

**URL:** https://discuss.elastic.co/t/delete-by-query-limitation/198028
**Category:** Elasticsearch
**Created:** [September 4, 2019, 10:32am UTC](https://discuss.elastic.co/t/delete-by-query-limitation/198028 "2019-09-04T10:32:21Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![tom\_es](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tom_es/32/53584_2.png) [@tom\_es](https://discuss.elastic.co/u/tom_es)
#### Post date: [September 4, 2019, 10:32am UTC](https://discuss.elastic.co/t/delete-by-query-limitation/198028/1 "2019-09-04T10:32:21Z")

</div>

Hi,  
we have an index with 50 Million docs, and from time to time we would like to purge some of it's data.

in order to do that, we would like to use "delete by" query.  
I was trying to understand what are the limitation of delete by query - meaning, is there a recommended threshold for the amount of documents that should be purged?

as out filter, in some cases, can potentially lead to a deletion of 10 million documents.  
is that a reasonable amount to delete? or it is recommended to work with some smaller bulks?

Thanks,  
Tom

---

<div class="post-metadata">

### Author: ![HenningAndersen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/henningandersen/32/48188_2.png) [@HenningAndersen](https://discuss.elastic.co/u/HenningAndersen)
#### Post date: [September 4, 2019, 10:47am UTC](https://discuss.elastic.co/t/delete-by-query-limitation/198028/2 "2019-09-04T10:47:13Z")

</div>

Hi @tom_es,

delete-by-query will delete documents in batches, so can handle a large amount of docs in one go.

The primary potential drawbacks of doing a large amount of docs in one go are:

1. The underlying scroll search will retain resources on the nodes containing the shard copy used for the search (one per shard). It will prevent deleting the old segments necessary to provide a snapshot of data until done.
2. If a data node or the node running delete-by-query is restarted, the process will fail.

10 million dodcs does sound reasonable to do in one go, though your mileage may vary depending on hardware and other activity.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 2, 2019, 10:47am UTC](https://discuss.elastic.co/t/delete-by-query-limitation/198028/3 "2019-10-02T10:47:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
