According to ES documentation document indexing/deletion happens as follows: Request received at one of the nodes. Request forwarded to the document's primary shard. The operation performed on the primary shard and parallel requests sent to replica nodes. Primary shard node waits for a response fr…

Hi @Rahul_Kumar3 , I think the missing piece to make this safe is a refresh. Delete by query basically does a search for the objects to delete and then deletes them with version conflict checking. Without a _refresh in between, the search done by _delete_by_query might return the old version of the…

Hi @HenningAndersen , So _delete_by_query basically searches for the documents to delete and then deletes them one by one. And a version conflict occurs if one or more of the documents gets update in between the time when the search was completed and the delete operation was started. But as I said,…

Hi @Rahul_Kumar3 , your summary is close to correct :slight_smile: a successful creation/updation does not imply that that the data is successfully persisted across the primary and replica shards The translog really resides on the primary and replica shards. So before Elasticsearch sends back a …

Hi @HenningAndersen , I am confused a bit here. You are saying that translog is fsynced before responding for a request by default. And according to this document, An Elasticsearch flush is the process of performing a Lucene commit and starting a new translog. . This would mean that each document is…

I believe this is the sequence of events: Request received at one of the nodes. Request forwarded to the document's primary shard. The operation performed on the primary shard and parallel requests sent to replica nodes. The translog is fsynced on primary and replica shards which makes it persiste…

This sequence of events does make sense. But according to this document, synced flush (fsync) is a special kind of flush which performs a normal flush, then adds a generated unique marker (sync_id) to all shards. And according to this document, an Elasticsearch flush is the process of performing …

It happens during refresh. A synced flush is a special operation and should not be confused with the fsyncing of the translog that occurs per request. In the flow I outlined above there would be no synced flush. Only if the API was explicitly called or the shard was idle for a period of time would …

Thanks @Christian_Dahlqvist , That does clears it for me.

Discuss the Elastic Stack

Elasticsearch delete_by_query 409 version conflict

Elastic Stack Elasticsearch

Topic		Replies	Views
Version conflict always on _delete_from_query Elasticsearch	7	2836	July 27, 2018
Version Conflict while using delete_by_query Elasticsearch	6	27422	November 22, 2017
Delete by Query and Refresh Interval Elasticsearch	7	3178	November 13, 2020
Version_conflict when trying to delete documents using _delete_by_query API Elasticsearch	1	757	April 14, 2023
409 - version conflict - delete_by_query (API request using C# NEST) Elasticsearch language-clients	7	952	November 15, 2023

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Elasticsearch delete_by_query 409 version conflict

Related topics