Patch for RabbitMQ river


(Damien Hardy) #1

Hello,

I join a patch to my mail to make HA Queue on RabbitMQ available for
Elasticsearch.
http://www.elasticsearch.org/guide/reference/river/rabbitmq.html

Based on the new repo
https://github.com/elasticsearch/elasticsearch-river-rabbitmq (congrats for
the good job Shay :slight_smile:

Reading this : http://www.rabbitmq.com/ha.html *

  • */!\* big warn only "x-ha-policy" : "all" is fonctionnal */!\*
    

see
http://old.nabble.com/HA-queue-seems-never-to-get-fully-created-td32529015.html#a32582000

Config for HA Queue should be :
curl -XPUT 'localhost:9200/_river/my_river/_meta' -d '{
"type" : "rabbitmq",
"rabbitmq" : {
"host" : "localhost",
"port" : 5672,
"user" : "guest",
"pass" : "guest",
"vhost" : "/",
"queue" : "elasticsearch",
"exchange" : "elasticsearch",
"routing_key" : "elasticsearch",
"exchange_type" : "direct",
"exchange_durable" : true,
"queue_durable" : true,
"queue_auto_delete" : false,
"args" : { "x-ha-policy" : "all" }
},
"index" : {
"bulk_size" : 100,
"bulk_timeout" : "10ms",
"ordered" : false
}
}

And should support future "x-ha-policy" : "nodes" (when bug will be closed
on RabbitMQ) with :
curl -XPUT 'localhost:9200/_river/my_river/_meta' -d '{
"type" : "rabbitmq",
"rabbitmq" : {
"host" : "localhost",
"port" : 5672,
"user" : "guest",
"pass" : "guest",
"vhost" : "/",
"queue" : "elasticsearch",
"exchange" : "elasticsearch",
"routing_key" : "elasticsearch",
"exchange_type" : "direct",
"exchange_durable" : true,
"queue_durable" : true,
"queue_auto_delete" : false,
"args" : {
"x-ha-policy" : "nodes",
"x-ha-policy-params" : [ "rabbit1@localhost",
"rabbit2@localhost" ]
}
},
"index" : {
"bulk_size" : 100,
"bulk_timeout" : "10ms",
"ordered" : false
}
}

Cheers,

--
Damien


(Shay Banon) #2

Heya, thanks for the work!. Can you send a pull request with the change?

On Tue, Dec 6, 2011 at 5:52 PM, Damien Hardy damienhardy.bal@gmail.comwrote:

Hello,

I join a patch to my mail to make HA Queue on RabbitMQ available for
Elasticsearch.
http://www.elasticsearch.org/guide/reference/river/rabbitmq.html

Based on the new repo
https://github.com/elasticsearch/elasticsearch-river-rabbitmq (congrats
for the good job Shay :slight_smile:

Reading this : http://www.rabbitmq.com/ha.html *

  • */!\* big warn only "x-ha-policy" : "all" is fonctionnal */!\*
    

see
http://old.nabble.com/HA-queue-seems-never-to-get-fully-created-td32529015.html#a32582000

Config for HA Queue should be :
curl -XPUT 'localhost:9200/_river/my_river/_meta' -d '{
"type" : "rabbitmq",
"rabbitmq" : {
"host" : "localhost",
"port" : 5672,
"user" : "guest",
"pass" : "guest",
"vhost" : "/",
"queue" : "elasticsearch",
"exchange" : "elasticsearch",
"routing_key" : "elasticsearch",
"exchange_type" : "direct",
"exchange_durable" : true,
"queue_durable" : true,
"queue_auto_delete" : false,
"args" : { "x-ha-policy" : "all" }
},
"index" : {
"bulk_size" : 100,
"bulk_timeout" : "10ms",
"ordered" : false
}
}

And should support future "x-ha-policy" : "nodes" (when bug will be closed
on RabbitMQ) with :
curl -XPUT 'localhost:9200/_river/my_river/_meta' -d '{
"type" : "rabbitmq",
"rabbitmq" : {
"host" : "localhost",
"port" : 5672,
"user" : "guest",
"pass" : "guest",
"vhost" : "/",
"queue" : "elasticsearch",
"exchange" : "elasticsearch",
"routing_key" : "elasticsearch",
"exchange_type" : "direct",
"exchange_durable" : true,
"queue_durable" : true,
"queue_auto_delete" : false,
"args" : {
"x-ha-policy" : "nodes",
"x-ha-policy-params" : [ "rabbit1@localhost",
"rabbit2@localhost" ]
}
},
"index" : {
"bulk_size" : 100,
"bulk_timeout" : "10ms",
"ordered" : false
}
}

Cheers,

--
Damien


(Damien Hardy) #3

Hello,

pull request applied :

Good news :slight_smile: RabbitMQ 2.7.0 is out
http://old.nabble.com/RabbitMQ-2.7.0-is-released-ts32810200.html
And bugfix so :

  • could not declare a mirrored queue with a policy of "nodes" and an
    explicit
    list of node names

Cheers,

--
Damien


(Shay Banon) #4

cool, upgraded to 2.7.0 as well as pulling your change.

On Wed, Dec 7, 2011 at 12:42 PM, Damien Hardy damienhardy.bal@gmail.comwrote:

Hello,

pull request applied :
https://github.com/elasticsearch/elasticsearch-river-rabbitmq/pull/1

Good news :slight_smile: RabbitMQ 2.7.0 is out
http://old.nabble.com/RabbitMQ-2.7.0-is-released-ts32810200.html
And bugfix so :

  • could not declare a mirrored queue with a policy of "nodes" and an
    explicit
    list of node names

Cheers,

--
Damien


(Shay Banon) #5

btw, tell me if you want 1.1.0 release, its simple now that its outside of
elasticsearch :).

On Wed, Dec 7, 2011 at 5:49 PM, Shay Banon kimchy@gmail.com wrote:

cool, upgraded to 2.7.0 as well as pulling your change.

On Wed, Dec 7, 2011 at 12:42 PM, Damien Hardy damienhardy.bal@gmail.comwrote:

Hello,

pull request applied :
https://github.com/elasticsearch/elasticsearch-river-rabbitmq/pull/1

Good news :slight_smile: RabbitMQ 2.7.0 is out
http://old.nabble.com/RabbitMQ-2.7.0-is-released-ts32810200.html
And bugfix so :

  • could not declare a mirrored queue with a policy of "nodes" and an
    explicit
    list of node names

Cheers,

--
Damien


(Damien Hardy) #6

I don't care. It's yours :slight_smile:

Sounds good to me btw.

2011/12/7 Shay Banon kimchy@gmail.com

btw, tell me if you want 1.1.0 release, its simple now that its outside of
elasticsearch :).

On Wed, Dec 7, 2011 at 5:49 PM, Shay Banon kimchy@gmail.com wrote:

cool, upgraded to 2.7.0 as well as pulling your change.

On Wed, Dec 7, 2011 at 12:42 PM, Damien Hardy damienhardy.bal@gmail.comwrote:

Hello,

pull request applied :
https://github.com/elasticsearch/elasticsearch-river-rabbitmq/pull/1

Good news :slight_smile: RabbitMQ 2.7.0 is out
http://old.nabble.com/RabbitMQ-2.7.0-is-released-ts32810200.html
And bugfix so :

  • could not declare a mirrored queue with a policy of "nodes" and an
    explicit
    list of node names

Cheers,

--
Damien


(system) #7