Little help/clarification about RabbitMQ plugin


(Paul Smith) #1

just checking this approach will work. If I install the RabbitMQ river ala
the ES docs with that funky bit of curl, that I can use the 2.2.0 version of
the amqp-client dependency in a project ala Maven:

com.rabbitmq amqp-client 2.2.0

and then simply publish messages containing JSON text ala the bulk API.

That's the way I read the River docs, I think this is right.... ?

In a clustered ES instance, I understand the Rivers are singletons, so will
only be on one node. From a client application sending messages, how will
it know which host to send the RabbitMQ messages to exactly.. ?

Or is the proper way to setup a RabbitMQ server on some host 'RabbitMQHost'
and have the river singleton just be the 'consumer' client of the queue ?
(so we have an ES process, as well as separate RabbitMQ process perhaps
sharing the same hardware, perhaps not).

I'd just like clarity whether the river is running as just the consumer
client, or is actually running an embedded MQ server and consumer client.

thanks,

Paul


(Rich Kroll) #2

I am not an expert, but I did a proof of concept with rabbit and es, so
anyone step in if I miss anything as this is from memory. The rabbit mq
plugin acts as a consumer of a given queue. In our POC, we wanted to queue
and persist each message for guaranteed delivery, so each es node had a
rabbit mq server running locally, with the plugin consuming from it. Our
clients could then be directed to the load balancer, and deliver their
message to any server in our cluster.
On Feb 28, 2011 11:44 PM, "Paul Smith" tallpsmith@gmail.com wrote:


(Shay Banon) #3

The rabbitmq river is a singleton in the cluster, and it subscribes to a queue that it then pulls bulk format requests from, yes. So, you need to have a rabbitmq "server" (and not sure how you make it highly available...) that the river will pull the requests from.
On Tuesday, March 1, 2011 at 2:36 PM, Rich Kroll wrote:

I am not an expert, but I did a proof of concept with rabbit and es, so anyone step in if I miss anything as this is from memory. The rabbit mq plugin acts as a consumer of a given queue. In our POC, we wanted to queue and persist each message for guaranteed delivery, so each es node had a rabbit mq server running locally, with the plugin consuming from it. Our clients could then be directed to the load balancer, and deliver their message to any server in our cluster.
On Feb 28, 2011 11:44 PM, "Paul Smith" tallpsmith@gmail.com wrote:


(system) #4