# BulkProcessor + NoNodeAvailableException - things to avoid **URL:** \ **Category:** Elasticsearch\ **Created:** [September 22, 2013, 9:14pm UTC](https://discuss.elastic.co/t/bulkprocessor-nonodeavailableexception-things-to-avoid/13716 "2013-09-22T21:14:18Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Abhishek\_Sanoujam](https://avatars.discourse-cdn.com/v4/letter/a/3da27b/32.png) [@Abhishek\_Sanoujam](https://discuss.elastic.co/u/Abhishek_Sanoujam)\ **Post date:** [September 22, 2013, 9:14pm UTC](https://discuss.elastic.co/t/bulkprocessor-nonodeavailableexception-things-to-avoid/13716/1 "2013-09-22T21:14:18Z")
Using BulkProcessor and rethrowing exception from the afterBulk() method should not be recommended at all. Currently, doing so results in closing down the transport and resulting in NoNodeAvailableException's subsequently. We learnt it the hard way, and the workaround/fix right now is to NOT throw exceptions. It would be good if there can be a best practices guide for BulkProcessor, at the least, it should be mentioned in javadoc to avoid throwing exceptions from the listener itself. I've filed an issue that demonstrates the problem with a unit test here: > An example BulkProcessor.Listener would be: ``` private static class BulkProcessorProblematicListener implements ``` BulkProcessor.Listener { @Override public void beforeBulk(long executionId, BulkRequest request) { ``` } @Override public void afterBulk(long executionId, BulkRequest request, ``` BulkResponse response) { if (response.hasFailures()) { throw new RuntimeException("Failure in response - " + response.buildFailureMessage()); } } ``` @Override public void afterBulk(long executionId, BulkRequest request, ``` Throwable failure) { // throwing exception here makes the transport close resulting in NoNodeAvailableException subsequently throw new RuntimeException("Caught exception in bulk: " + request + ", failure: " + failure, failure); } } ## -- Cheers, Abhishek -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [September 23, 2013, 7:29am UTC](https://discuss.elastic.co/t/bulkprocessor-nonodeavailableexception-things-to-avoid/13716/2 "2013-09-23T07:29:12Z")
I do not think there is a problem since the methods are not declared with any exceptions that should be thrown. Here are some best practices: - check if your bulk feed is driven by interval or by doc volume and adjust the BulkProcessor setup accordingly. The default is 5mb (doc volume) and the flush interval is 5 seconds. - check each BulkResponse in the BulkProcessor.Listener for failures. Reindex the corresponding document. If you want to stop bulk indexing, set a volatile boolean error variable that can be checked before a new bulk is submitted. - after adding the last bulk request, wait for at least 5 seconds before closing the BulkProcessor, so a flush is triggered for the last bulk. - do not use more than one thread to add bulk requests, since the underlying list of requests is not threadsafe. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Abhishek\_Sanoujam](https://avatars.discourse-cdn.com/v4/letter/a/3da27b/32.png) [@Abhishek\_Sanoujam](https://discuss.elastic.co/u/Abhishek_Sanoujam)\ **Post date:** [September 23, 2013, 8:26am UTC](https://discuss.elastic.co/t/bulkprocessor-nonodeavailableexception-things-to-avoid/13716/3 "2013-09-23T08:26:05Z")
Documenting it up would definitely help - something that results in getting the transport closed is pretty bad. Also, it would be better if elasticsearch can internally take care of mistakes that user can make and make it more robust by not closing down the transport when they throw exceptions unknowingly. I'd argue that as a bug unless its clearly mentioned in the javadoc. Also, some confusion wrt the points u mentioned below inlined. On 9/23/13 12:59 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > I do not think there is a problem since the methods are not declared > with any exceptions that should be thrown. > > Here are some best practices: > > - check if your bulk feed is driven by interval or by doc volume and > adjust the BulkProcessor setup accordingly. The default is 5mb (doc > volume) and the flush interval is 5 seconds. > > - check each BulkResponse in the BulkProcessor.Listener for failures. > Reindex the corresponding document. If you want to stop bulk indexing, > set a volatile boolean error variable that can be checked before a new > bulk is submitted. > > - after adding the last bulk request, wait for at least 5 seconds > before closing the BulkProcessor, so a flush is triggered for the last > bulk. Correct me if I'm wrong, but this looks totally unnecessary. From javadoc of close(): "Closes the processor. If flushing by time is enabled, then its shutdown. Any remaining bulk actions are flushed." Also looking at the code, it does seem like the thread calling close() would flush any remaining items. > - do not use more than one thread to add bulk requests, since the > underlying list of requests is not threadsafe. This came as a surprise. This is in total contrast of what 'concurrentRequests' in BulkProcessor.Builder suggests. All the add(), execute() and close() methods in bulkProcessor also seem to be synchronized - don't understand how underlying bulkRequest not being thread safe matters. Are you sure this is recommended? > Jörg > > -- > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ## -- Cheers, Abhishek -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:15am UTC](https://discuss.elastic.co/t/bulkprocessor-nonodeavailableexception-things-to-avoid/13716/4 "2017-07-06T02:15:10Z")