# Queue capacity and EsRejectedExecutionException leads to loss of data **URL:** \ **Category:** Elasticsearch\ **Created:** [February 26, 2014, 1:48pm UTC](https://discuss.elastic.co/t/queue-capacity-and-esrejectedexecutionexception-leads-to-loss-of-data/16031 "2014-02-26T13:48:55Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Thomas\_Bolis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thomas_bolis/32/700_2.png) [@Thomas\_Bolis](https://discuss.elastic.co/u/Thomas_Bolis)\ **Post date:** [February 26, 2014, 1:48pm UTC](https://discuss.elastic.co/t/queue-capacity-and-esrejectedexecutionexception-leads-to-loss-of-data/16031/1 "2014-02-26T13:48:55Z")
Hi, We have installed the RabbitMQ river plugin to pull data from our Queue and adding them to ES. The thing is that at some point we are receiving the following exception and we have as a result to _lose data_. [1775]: index [events-idx], type [click], id > [3f6e4604146b435aabcf4ea5a493fd32], message > [EsRejectedExecutionException[rejected execution (queue capacity 50) on > org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@12843ca2]] We have changed the configuration of queue size to 1000 and the problem disappeared. My question is that is there any configuration/way to tell ES to instead of throwing this exception and discarding the document to wait for available resources (with the corresponding performance impact)? Thanks Thomas -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4e11ce4e-26d4-4a53-a53e-0ba89bde4605%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4e11ce4e-26d4-4a53-a53e-0ba89bde4605%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 26, 2014, 2:29pm UTC](https://discuss.elastic.co/t/queue-capacity-and-esrejectedexecutionexception-leads-to-loss-of-data/16031/2 "2014-02-26T14:29:21Z")
I think you should open an issue. It's to me somehow related to this issue: [https://github.com/elasticsearch/elasticsearch-river-rabbitmq/issues/47](https://github.com/elasticsearch/elasticsearch-river-rabbitmq/issues/47) and its pending PR: [https://github.com/elasticsearch/elasticsearch-river-rabbitmq/pull/48](https://github.com/elasticsearch/elasticsearch-river-rabbitmq/pull/48) I think we should have a new option to let the user decides if if want to ack messages in case of failure or leave them in queue. My 2 cents -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr Le 26 février 2014 à 14:49:00, Thomas ([thomas.bolis@gmail.com](mailto:thomas.bolis@gmail.com)) a écrit: Hi, We have installed the RabbitMQ river plugin to pull data from our Queue and adding them to ES. The thing is that at some point we are receiving the following exception and we have as a result to lose data. [1775]: index [events-idx], type [click], id [3f6e4604146b435aabcf4ea5a493fd32], message [EsRejectedExecutionException[rejected execution (queue capacity 50) on org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@12843ca2]] We have changed the configuration of queue size to 1000 and the problem disappeared. My question is that is there any configuration/way to tell ES to instead of throwing this exception and discarding the document to wait for available resources (with the corresponding performance impact)? Thanks Thomas -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4e11ce4e-26d4-4a53-a53e-0ba89bde4605%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4e11ce4e-26d4-4a53-a53e-0ba89bde4605%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/etPan.530dfa41.22221a70.5e46%40MacBook-Air-de-David.local](https://groups.google.com/d/msgid/elasticsearch/etPan.530dfa41.22221a70.5e46%40MacBook-Air-de-David.local). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Thomas\_Bolis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thomas_bolis/32/700_2.png) [@Thomas\_Bolis](https://discuss.elastic.co/u/Thomas_Bolis)\ **Post date:** [February 26, 2014, 4:00pm UTC](https://discuss.elastic.co/t/queue-capacity-and-esrejectedexecutionexception-leads-to-loss-of-data/16031/3 "2014-02-26T16:00:07Z")
Thanks David, So this is a rabbitMQRiver issue, is there a need to open a separate issue? (Never done the procedure, will look this one) Thomas On Wednesday, 26 February 2014 15:48:55 UTC+2, Thomas wrote: > Hi, > > We have installed the RabbitMQ river plugin to pull data from our Queue > and adding them to ES. The thing is that at some point we are receiving the > following exception and we have as a result to _lose data_. > > [1775]: index [events-idx], type [click], id > > > [3f6e4604146b435aabcf4ea5a493fd32], message > > [EsRejectedExecutionException[rejected execution (queue capacity 50) on > > org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@12843ca2]] > > We have changed the configuration of queue size to 1000 and the problem > disappeared. > > My question is that is there any configuration/way to tell ES to instead > of throwing this exception and discarding the document to wait for > available resources (with the corresponding performance impact)? > > Thanks > > Thomas -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/03dcb0ea-2b6a-478b-b678-f52ecbc09298%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03dcb0ea-2b6a-478b-b678-f52ecbc09298%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 26, 2014, 4:15pm UTC](https://discuss.elastic.co/t/queue-capacity-and-esrejectedexecutionexception-leads-to-loss-of-data/16031/4 "2014-02-26T16:15:20Z")
I think that adding a comment into the existing issue would be fine. -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr Le 26 février 2014 à 17:00:12, Thomas ([thomas.bolis@gmail.com](mailto:thomas.bolis@gmail.com)) a écrit: Thanks David, So this is a rabbitMQRiver issue, is there a need to open a separate issue? (Never done the procedure, will look this one) Thomas On Wednesday, 26 February 2014 15:48:55 UTC+2, Thomas wrote: Hi, We have installed the RabbitMQ river plugin to pull data from our Queue and adding them to ES. The thing is that at some point we are receiving the following exception and we have as a result to lose data. [1775]: index [events-idx], type [click], id [3f6e4604146b435aabcf4ea5a493fd32], message [EsRejectedExecutionException[rejected execution (queue capacity 50) on org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@12843ca2]] We have changed the configuration of queue size to 1000 and the problem disappeared. My question is that is there any configuration/way to tell ES to instead of throwing this exception and discarding the document to wait for available resources (with the corresponding performance impact)? Thanks Thomas -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/03dcb0ea-2b6a-478b-b678-f52ecbc09298%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03dcb0ea-2b6a-478b-b678-f52ecbc09298%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/etPan.530e1318.440badfc.5e46%40MacBook-Air-de-David.local](https://groups.google.com/d/msgid/elasticsearch/etPan.530e1318.440badfc.5e46%40MacBook-Air-de-David.local). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:47am UTC](https://discuss.elastic.co/t/queue-capacity-and-esrejectedexecutionexception-leads-to-loss-of-data/16031/5 "2017-07-06T01:47:07Z")