Delete using scroll/scan API and Bulk request Example

wajahat_Javaid · June 29, 2015, 8:17am

Since "Delete by Query" has been deprecated as a method to delete documents, I want to implement the work around to use scan/scroll API and Bulk Delete request.

Can Someone help me with an example, as I am not sure how to use scroll_id, returned by my search, in bulk delete request (or) there will be multiple bulk delete requests.

Clinton_Gormley · June 30, 2015, 8:07am

Hi @wajahat_Javaid

We have reimplemented delete-by-query using scan/scroll/bulk for you in a plugin, that you will be able to install with Elasticsearch 2.0. See https://github.com/elastic/elasticsearch/blob/master/docs/plugins/delete-by-query.asciidoc

wajahat_Javaid · June 30, 2015, 1:21pm

Thanks for the reply, The plugin would indeed come in very handy, but I am in need for this operation at present and the plugin would be compatible with elasticsearch 2.0,for now, I would need help in executing delete requests using scan/scroll API.

voipoclay · November 4, 2015, 1:52am

Does this need to be installed on every master only nodes as well as data nodes?

sushmitharaos · August 25, 2016, 3:51am

Reviving this old thread. The plugin seems to not be available anymore. I want to delete all documents of a type and would like to implement it using scroll API and bulk delete. Could someone please help me with an example of how I could achieve this? Is there is a better alternative to do the same?

magnusbaeck · August 25, 2016, 5:33am

The plugin seems to not be available anymore.

What makes you say that? Which version of ES are you using?

sushmitharaos · August 25, 2016, 6:15am

My bad! I am using version 2.3 of ES and I could install and use the plugin. Just the link mentioned in the previous post was incorrect. Here is the right link :

github.com

elastic/elasticsearch/blob/2.3/docs/plugins/delete-by-query.asciidoc

[[plugins-delete-by-query]]
=== Delete By Query Plugin

The delete-by-query plugin adds support for deleting all of the documents
(from one or more indices) which match the specified query. It is a
replacement for the problematic _delete-by-query_ functionality which has been
removed from Elasticsearch core.

Internally, it uses {ref}/search-request-scroll.html[Scroll]
and {ref}/docs-bulk.html[Bulk] APIs to delete documents in an efficient and
safe manner. It is slower than the old _delete-by-query_ functionality, but
fixes the problems with the previous implementation.

To understand more about why we removed delete-by-query from core and about
the semantics of the new implementation, see
<<delete-by-query-plugin-reason>>.

[TIP]
============================================
Queries which match large numbers of documents may run for a long time,

This file has been truncated. show original

OR
https://www.elastic.co/guide/en/elasticsearch/plugins/current/delete-by-query-usage.html