# Infinite bulk-rejection retries via es.batch.write.retry.count = -1

**URL:** <https://discuss.elastic.co/t/infinite-bulk-rejection-retries-via-es-batch-write-retry-count-1/140810>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [July 19, 2018, 9:51pm UTC](https://discuss.elastic.co/t/infinite-bulk-rejection-retries-via-es-batch-write-retry-count-1/140810 "2018-07-19T21:51:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sumitsu](https://avatars.discourse-cdn.com/v4/letter/s/50afbb/32.png) [@sumitsu](https://discuss.elastic.co/u/sumitsu)\
**Post date:** [July 19, 2018, 9:51pm UTC](https://discuss.elastic.co/t/infinite-bulk-rejection-retries-via-es-batch-write-retry-count-1/140810/1 "2018-07-19T21:51:33Z")

</div>

The [ES-Hadoop configuration documentation](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#configuration-serialization) states that setting `es.batch.write.retry.count` to a negative number will direct the ES-Hadoop connector to retry infinitely upon bulk-rejection failures. Could someone advise as to where/how infinite retries are implemented?

Was investigating a Spark app which was failing with ["Could not write all entries for bulk operation" EsHadoopException](https://github.com/elastic/elasticsearch-hadoop/blob/134d4c5/mr/src/main/java/org/elasticsearch/hadoop/rest/bulk/BulkProcessor.java#L465), which would seem to be impossible if the framework were retrying bulk-rejection failures infinitely.

Tracing the [`es.batch.write.retry.count`](https://github.com/elastic/elasticsearch-hadoop/blob/a0d9604/mr/src/main/java/org/elasticsearch/hadoop/cfg/ConfigurationOptions.java#L87) configuration option appears to show that it is only ever referenced at [this location in `HttpRetryHandler`](https://github.com/elastic/elasticsearch-hadoop/blob/d0e6155/mr/src/main/java/org/elasticsearch/hadoop/rest/bulk/handler/impl/HttpRetryHandler.java#L57). In turn, that implementation would appear to result in bulk-rejection failures _never_ retrying rather than retrying indefinitely.

Is there some other explanation I haven't considered? Or is there an actual bug / inconsistency with the documentation for that configuration setting?

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [July 25, 2018, 3:00pm UTC](https://discuss.elastic.co/t/infinite-bulk-rejection-retries-via-es-batch-write-retry-count-1/140810/2 "2018-07-25T15:00:10Z")

</div>

I'd agree with you here on the inconsistency. It looks like this is a bug that crept in during the rework of the bulk failure handling code when we added failure handlers to the API. I've opened [https://github.com/elastic/elasticsearch-hadoop/issues/1178](https://github.com/elastic/elasticsearch-hadoop/issues/1178) to track this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 22, 2018, 3:00pm UTC](https://discuss.elastic.co/t/infinite-bulk-rejection-retries-via-es-batch-write-retry-count-1/140810/3 "2018-08-22T15:00:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.