# Response of asynchronous bulk request when a node dies

**URL:** <https://discuss.elastic.co/t/response-of-asynchronous-bulk-request-when-a-node-dies/251422>\
**Category:** Elasticsearch\
**Created:** [October 8, 2020, 11:00am UTC](https://discuss.elastic.co/t/response-of-asynchronous-bulk-request-when-a-node-dies/251422 "2020-10-08T11:00:16Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![harisk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harisk/32/75997_2.png) [@harisk](https://discuss.elastic.co/u/harisk)\
**Post date:** [October 8, 2020, 11:00am UTC](https://discuss.elastic.co/t/response-of-asynchronous-bulk-request-when-a-node-dies/251422/1 "2020-10-08T11:00:16Z")

</div>

I have trouble with the response of asynchronous bulk request when a node dies.

For example, I run **1 master** node, **2 data** nodes, and **2 client** nodes (coordinating nodes).

I test on the index with **2 primary** shards **without replication**.

I **stop 1 data** node, and I execute an async bulk request including a few documents.

I'm confused that the callback for the async request is `onFailure(Exception e)`, not `onResponse(BulkResponse bulkResponse)`. Because I'd to know which documents are indexed successfully and which documents fail. I think if the callback is `onResponse(BulkResponse bulkResponse)`, I can filter failure documents to retry.

Can you explain the problem if I had any mistakes?

Thanks!

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 11, 2020, 7:51am UTC](https://discuss.elastic.co/t/response-of-asynchronous-bulk-request-when-a-node-dies/251422/2 "2020-10-11T07:51:33Z")

</div>

If you only have primary shards and no replicas, losing one node will make the index red as one primary is missing. This as far as I remember will prevent all indexing to the index, not only for documents that would have gone to the missing primary shard.

---

<div class="post-metadata">

**Author:** ![harisk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harisk/32/75997_2.png) [@harisk](https://discuss.elastic.co/u/harisk)\
**Post date:** [October 12, 2020, 2:01am UTC](https://discuss.elastic.co/t/response-of-asynchronous-bulk-request-when-a-node-dies/251422/3 "2020-10-12T02:01:04Z")

</div>

After I stop one data node, my index goes red. However, when I put new data, my index continues to add new documents.

> 2 data node work  
> **![](https://us1.discourse-cdn.com/elastic/original/3X/7/b/7b0d1994548300659e082e1cc7325922b2e1dea9.png)**

> 1 data node dies  
> **![](https://us1.discourse-cdn.com/elastic/original/3X/c/3/c37e928ea440e5afb549336d6b61f760e8172517.png)**

> index new documents  
> **![](https://us1.discourse-cdn.com/elastic/original/3X/9/9/9953b3369e394c10927b182eba5cd8ab8f167233.png)**

I've also tried many times but the ES still index new documents (timeout for new documents routed to die node).

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 12, 2020, 3:32am UTC](https://discuss.elastic.co/t/response-of-asynchronous-bulk-request-when-a-node-dies/251422/4 "2020-10-12T03:32:07Z")

</div>

That is not what I expected, so it seems I might be wrong. Would still recommend always running with a replica configured to prevent data loss.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 9, 2020, 3:32am UTC](https://discuss.elastic.co/t/response-of-asynchronous-bulk-request-when-a-node-dies/251422/5 "2020-11-09T03:32:36Z")

</div>

Closed on November 9, 2020, 3:32am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
