# About RabbitMQ River **URL:** \ **Category:** Elasticsearch\ **Created:** [September 13, 2011, 7:16pm UTC](https://discuss.elastic.co/t/about-rabbitmq-river/5374 "2011-09-13T19:16:17Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Frederic](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frederic](https://discuss.elastic.co/u/Frederic)\ **Post date:** [September 13, 2011, 7:16pm UTC](https://discuss.elastic.co/t/about-rabbitmq-river/5374/1 "2011-09-13T19:16:17Z")
I've read the [http://www.elasticsearch.org/guide/reference/river/rabbitmq.html](http://www.elasticsearch.org/guide/reference/river/rabbitmq.html) page but have some questions though: - What would happen in case of errors when indexing a bulk of jsons? is that info put in the queue again (not ACK'ed)? - Although every River is a singleton, does it make sense to configure 2 rivers pointing to the same queue in order to process it in parallel? ---
**Author:** ![Alberto\_Paro\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alberto_paro_2/32/1137_2.png) [@Alberto\_Paro\_2](https://discuss.elastic.co/u/Alberto_Paro_2)\ **Post date:** [September 13, 2011, 7:31pm UTC](https://discuss.elastic.co/t/about-rabbitmq-river/5374/2 "2011-09-13T19:31:18Z")
Sent from my iPhone On 13/set/2011, at 21:16, Frederic [focampo.br@gmail.com](mailto:focampo.br@gmail.com) wrote: > I've read the [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/river/rabbitmq.html) > page but have some questions though: > > - What would happen in case of errors when indexing a bulk of jsons? > is that info put in the queue again (not ACK'ed)? you lost all the items you have ack'ed > - Although every River is a singleton, does it make sense to configure > 2 rivers pointing to the same queue in order to process it in parallel? > yes to increase parallelism you can activate several rivers on the same queue: rabbitmq grant unique delivery of a message. > hit: tune the bulk\_size ---
**Author:** ![Frederic](https://avatars.discourse-cdn.com/v4/letter/f/4491bb/32.png) [@Frederic](https://discuss.elastic.co/u/Frederic)\ **Post date:** [September 14, 2011, 5:44pm UTC](https://discuss.elastic.co/t/about-rabbitmq-river/5374/3 "2011-09-14T17:44:54Z")
Thanks Alberto for your answer. When you say: > you lost all the items you have ack'ed You mean that if the river takes 100 items to index and after having indexed 50 the server fails, the other 50 will be lost? I'm not sure how the Bulk Index process works, but if it is atomic, I guess that, in case of failure, none of the 100 items will be ack'ed and they will keep in the queue until the next try right? Gracias, On 13 sep, 16:31, Alberto Paro [alberto.p...@gmail.com](mailto:alberto.p...@gmail.com) wrote: > Sent from my iPhone > > On 13/set/2011, at 21:16, Frederic [focampo...@gmail.com](mailto:focampo...@gmail.com) wrote: > > > I've read thehttp://www.elasticsearch.org/guide/reference/river/rabbitmq.html > > page but have some questions though: > > > - What would happen in case of errors when indexing a bulk of jsons? > > is that info put in the queue again (not ACK'ed)? > > you lost all the items you have ack'ed > > > - Although every River is a singleton, does it make sense to configure > > 2 rivers pointing to the same queue in order to process it in parallel? > > yes to increase parallelism you can activate several rivers on the same queue: rabbitmq grant unique delivery of a message. > hit: tune the bulk\_size ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 15, 2011, 9:35am UTC](https://discuss.elastic.co/t/about-rabbitmq-river/5374/4 "2011-09-15T09:35:44Z")
Bulk indexing is not atomic, the ones that were indexed are indexed, the others are logged. When I wrote it, I added a TODO to possibly write the failed docs to an exception queue, but never got around to do it 🙂 : [https://github.com/elasticsearch/elasticsearch/blob/master/plugins/river/rabbitmq/src/main/java/org/elasticsearch/river/rabbitmq/RabbitmqRiver.java#L263](https://github.com/elasticsearch/elasticsearch/blob/master/plugins/river/rabbitmq/src/main/java/org/elasticsearch/river/rabbitmq/RabbitmqRiver.java#L263) . On Wed, Sep 14, 2011 at 8:44 PM, Frederic [focampo.br@gmail.com](mailto:focampo.br@gmail.com) wrote: > Thanks Alberto for your answer. When you say: > > > you lost all the items you have ack'ed > > You mean that if the river takes 100 items to index and after having > indexed 50 the server fails, the other 50 will be lost? I'm not sure > how the Bulk Index process works, but if it is atomic, I guess that, > in case of failure, none of the 100 items will be ack'ed and they will > keep in the queue until the next try right? > > Gracias, > > On 13 sep, 16:31, Alberto Paro [alberto.p...@gmail.com](mailto:alberto.p...@gmail.com) wrote: > > > Sent from my iPhone > > > > On 13/set/2011, at 21:16, Frederic [focampo...@gmail.com](mailto:focampo...@gmail.com) wrote: > > > > > I've read thehttp:// > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/river/rabbitmq.html) > > > page but have some questions though: > > > > > - What would happen in case of errors when indexing a bulk of jsons? > > > is that info put in the queue again (not ACK'ed)? > > > > you lost all the items you have ack'ed > > > > > - Although every River is a singleton, does it make sense to configure > > > 2 rivers pointing to the same queue in order to process it in parallel? > > > > yes to increase parallelism you can activate several rivers on the same > > queue: rabbitmq grant unique delivery of a message. > > hit: tune the bulk\_size ---
**Author:** ![Alberto\_Paro\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alberto_paro_2/32/1137_2.png) [@Alberto\_Paro\_2](https://discuss.elastic.co/u/Alberto_Paro_2)\ **Post date:** [September 17, 2011, 10:22am UTC](https://discuss.elastic.co/t/about-rabbitmq-river/5374/5 "2011-09-17T10:22:19Z")
Sent from my iPhone On 14/set/2011, at 19:44, Frederic [focampo.br@gmail.com](mailto:focampo.br@gmail.com) wrote: > Thanks Alberto for your answer. When you say: > > > you lost all the items you have ack'ed > > You mean that if the river takes 100 items to index and after having > indexed 50 the server fails, the other 50 will be lost? I'm not sure > how the Bulk Index process works, but if it is atomic, I guess that, > in case of failure, none of the 100 items will be ack'ed and they will > keep in the queue until the next try right? when the river riceive a message, it acks it and then It do the bulk. acked messages if the river dies on bulk you may lost these messagges. the problem is during bulk filling. > Gracias, > > On 13 sep, 16:31, Alberto Paro [alberto.p...@gmail.com](mailto:alberto.p...@gmail.com) wrote: > > > Sent from my iPhone > > > > On 13/set/2011, at 21:16, Frederic [focampo...@gmail.com](mailto:focampo...@gmail.com) wrote: > > > > > I've read thehttp://www.elasticsearch.org/guide/reference/river/rabbitmq.html > > > page but have some questions though: > > > > > - What would happen in case of errors when indexing a bulk of jsons? > > > is that info put in the queue again (not ACK'ed)? > > > > you lost all the items you have ack'ed > > > > > - Although every River is a singleton, does it make sense to configure > > > 2 rivers pointing to the same queue in order to process it in parallel? > > > > yes to increase parallelism you can activate several rivers on the same queue: rabbitmq grant unique delivery of a message. > > hit: tune the bulk\_size ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:54am UTC](https://discuss.elastic.co/t/about-rabbitmq-river/5374/6 "2017-07-06T03:54:13Z")