# ElasticSearch and RabbitMQ as data source **URL:** \ **Category:** Elasticsearch\ **Created:** [July 4, 2013, 9:06am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659 "2013-07-04T09:06:53Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![11124](https://avatars.discourse-cdn.com/v4/letter/1/b5a626/32.png) [@11124](https://discuss.elastic.co/u/11124)\ **Post date:** [July 4, 2013, 9:06am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/1 "2013-07-04T09:06:53Z")
Hi all, I am new to RabbitMQ and ElasticSearch. I am trying to create a setup where RabbitMQ collects messages from some sources and publish then in ElasticSearch for indexing. What I have done so far: Installed ElasticSearch 0.9.2 on Ubuntu 12.04 Created index named rabbit2 with these mappings: { "settings" : { "number\_of\_shards" : 5 }, "mappings" : { "logentry" : { "\_all" : {"enabled" : false}, "Metadata" : { "type" : "string", "index" : "not\_analyzed" }, "Categories" : {"type" : "string", "index" : "not\_analyzed" }, "ExtendedProperties" : { "type" : "object", "properties" : { "DocumentKey": {"type" : "string", "index" : "not\_analyzed"}, "Message": {"type" : "string", "index" : "analyzed"}, "ErrorCode": {"type" : "integer", "store" : "yes" }, "InnerMessage": {"type" : "string", "index": "analyzed"} } }, "Message": {"type" : "string", "index" : "not\_analyzed"}, "Timestamp": {"type" : "date", "format" : "date\_time" } } } } Installed RabbitMQ river and configured it with this settings { "type" : "rabbitmq", "rabbitmq" : { "host" : "192.168.106.11", "port" : 5672, "user" : "guest", "pass" : "guest", "vhost" : "playground", "queue" : "platform.flume.queue", "exchange" : "platform.flume.exchange", "routing\_key" : "flume", "exchange\_declare" : false, "exchange\_type" : "direct", "exchange\_durable" : false, "queue\_declare" : false, "queue\_bind" : true, "queue\_durable" : true, "queue\_auto\_delete" : false }, "index" : { "name": "rabbit2", "type" : "logentry", "bulk\_size" : 100, "bulk\_timeout" : "10ms", "ordered" : false } } ES log shows no error, RabbitMQ Web UI shows that there is a consumer connected with the IP address of the ES server RabbitMQ exchange and queue are predefined and already existing. I have published from RabbitMQ Web UI following message: { "create": { "\_index": "rabbit2", "\_type": "logentry", "\_id": 2 }} {"Metadata": ["PLATFORM-PL1","10.176.130.201;","Platform.CPBC.Container.exe","Platform.CPBC.Container","5584", null,"4020","0","Error"],"Categories": ["PLATFORM-PL1.platform.cpbc.data"],"ExtendedProperties": {"DocumentKey": "tenant-alerts\_1e5d2fa7-0e7a-4f02-b91a-cf09e13e66af", "Message": "Failed to process response: Not found", "ErrorCode": 1, "InnerMessage": "Failed to process response: Not found" }, "Message": "Document was not found.", "Timestamp": "2013-06-26T18:00:07.6122425Z"} Unfortunately the message is not consumed by ES and stays as "Unacknowledged". My question what I am doing wrong? Did I created wrong configuration or missed something? Is there somthing wrong with the message that I published? Thanks for your help in advance. best regards, nickolay kolev -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 4, 2013, 9:22am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/2 "2013-07-04T09:22:57Z")
## Anything in logs? David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 4 juil. 2013 à 11:06, Николай Колев [lem00na@gmail.com](mailto:lem00na@gmail.com) a écrit : > Hi all, > > I am new to RabbitMQ and Elasticsearch. I am trying to create a setup where RabbitMQ collects messages from some sources and publish then in Elasticsearch for indexing. > What I have done so far: > > Installed Elasticsearch 0.9.2 on Ubuntu 12.04 > Created index named rabbit2 with these mappings: > { > "settings" : { > "number\_of\_shards" : 5 > }, > "mappings" : { > "logentry" : { > "\_all" : {"enabled" : false}, > "Metadata" : { "type" : "string", "index" : "not\_analyzed" }, > "Categories" : {"type" : "string", "index" : "not\_analyzed" }, > "ExtendedProperties" : { > "type" : "object", > "properties" : { > "DocumentKey": {"type" : "string", "index" : "not\_analyzed"}, > "Message": {"type" : "string", "index" : "analyzed"}, > "ErrorCode": {"type" : "integer", "store" : "yes" }, > "InnerMessage": {"type" : "string", "index": "analyzed"} > } > }, > "Message": {"type" : "string", "index" : "not\_analyzed"}, > "Timestamp": {"type" : "date", "format" : "date\_time" } > } > } > } > > Installed RabbitMQ river and configured it with this settings > { > "type" : "rabbitmq", > "rabbitmq" : { > "host" : "192.168.106.11", > "port" : 5672, > "user" : "guest", > "pass" : "guest", > "vhost" : "playground", > "queue" : "platform.flume.queue", > "exchange" : "platform.flume.exchange", > "routing\_key" : "flume", > "exchange\_declare" : false, > "exchange\_type" : "direct", > "exchange\_durable" : false, > "queue\_declare" : false, > "queue\_bind" : true, > "queue\_durable" : true, > "queue\_auto\_delete" : false > }, > "index" : { > "name": "rabbit2", > "type" : "logentry", > "bulk\_size" : 100, > "bulk\_timeout" : "10ms", > "ordered" : false > } > } > > ES log shows no error, RabbitMQ Web UI shows that there is a consumer connected with the IP address of the ES server > RabbitMQ exchange and queue are predefined and already existing. > > I have published from RabbitMQ Web UI following message: > > { "create": { "\_index": "rabbit2", "\_type": "logentry", "\_id": 2 }} > {"Metadata": ["PLATFORM-PL1","10.176.130.201;","Platform.CPBC.Container.exe","Platform.CPBC.Container","5584", null,"4020","0","Error"],"Categories": ["PLATFORM-PL1.platform.cpbc.data"],"ExtendedProperties": {"DocumentKey": "tenant-alerts\_1e5d2fa7-0e7a-4f02-b91a-cf09e13e66af", "Message": "Failed to process response: Not found", "ErrorCode": 1, "InnerMessage": "Failed to process response: Not found" }, "Message": "Document was not found.", "Timestamp": "2013-06-26T18:00:07.6122425Z"} > > Unfortunately the message is not consumed by ES and stays as "Unacknowledged". > > My question what I am doing wrong? Did I created wrong configuration or missed something? > Is there somthing wrong with the message that I published? > > Thanks for your help in advance. > > best regards, > nickolay kolev > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![11124](https://avatars.discourse-cdn.com/v4/letter/1/b5a626/32.png) [@11124](https://discuss.elastic.co/u/11124)\ **Post date:** [July 4, 2013, 9:26am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/3 "2013-07-04T09:26:25Z")
Hi David, There were some errors while I was trying to figure what I have to do but now it seems that it is correct and works: [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses [[192.168.106.11:5672]], user [guest], vhost [playground] [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) Is there and option to make it more verbose? best regards, nickolay kolev 04 юли 2013, четвъртък, 12:22:57 UTC+3, David Pilato написа: > ## Anything in logs? > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 4 juil. 2013 à 11:06, Николай Колев \<[lem...@gmail.com](mailto:lem...@gmail.com) \\> a > écrit : > > Hi all, > > I am new to RabbitMQ and Elasticsearch. I am trying to create a setup > where RabbitMQ collects messages from some sources and publish then in > Elasticsearch for indexing. > What I have done so far: > > Installed Elasticsearch 0.9.2 on Ubuntu 12.04 > Created index named rabbit2 with these mappings: > { > "settings" : { > "number\_of\_shards" : 5 > }, > "mappings" : { > "logentry" : { > "\_all" : {"enabled" : false}, > "Metadata" : { "type" : "string", "index" : "not\_analyzed" }, > "Categories" : {"type" : "string", "index" : "not\_analyzed" }, > "ExtendedProperties" : { > "type" : "object", > "properties" : { > "DocumentKey": {"type" : "string", "index" : "not\_analyzed"}, > "Message": {"type" : "string", "index" : "analyzed"}, > "ErrorCode": {"type" : "integer", "store" : "yes" }, > "InnerMessage": {"type" : "string", "index": "analyzed"} > } > }, > "Message": {"type" : "string", "index" : "not\_analyzed"}, > "Timestamp": {"type" : "date", "format" : "date\_time" } > } > } > } > > Installed RabbitMQ river and configured it with this settings > { > "type" : "rabbitmq", > "rabbitmq" : { > "host" : "192.168.106.11", > "port" : 5672, > "user" : "guest", > "pass" : "guest", > "vhost" : "playground", > "queue" : "platform.flume.queue", > "exchange" : "platform.flume.exchange", > "routing\_key" : "flume", > "exchange\_declare" : false, > "exchange\_type" : "direct", > "exchange\_durable" : false, > "queue\_declare" : false, > "queue\_bind" : true, > "queue\_durable" : true, > "queue\_auto\_delete" : false > }, > "index" : { > "name": "rabbit2", > "type" : "logentry", > "bulk\_size" : 100, > "bulk\_timeout" : "10ms", > "ordered" : false > } > } > > ES log shows no error, RabbitMQ Web UI shows that there is a consumer > connected with the IP address of the ES server > RabbitMQ exchange and queue are predefined and already existing. > > I have published from RabbitMQ Web UI following message: > > { "create": { "\_index": "rabbit2", "\_type": "logentry", "\_id": 2 }} > {"Metadata": > ["PLATFORM-PL1","10.176.130.201;","Platform.CPBC.Container.exe","Platform.CPBC.Container","5584", > null,"4020","0","Error"],"Categories": > ["PLATFORM-PL1.platform.cpbc.data"],"ExtendedProperties": {"DocumentKey": > "tenant-alerts\_1e5d2fa7-0e7a-4f02-b91a-cf09e13e66af", "Message": "Failed to > process response: Not found", "ErrorCode": 1, "InnerMessage": "Failed to > process response: Not found" }, "Message": "Document was not found.", > "Timestamp": "2013-06-26T18:00:07.6122425Z"} > > Unfortunately the message is not consumed by ES and stays as > "Unacknowledged". > > My question what I am doing wrong? Did I created wrong configuration or > missed something? > Is there somthing wrong with the message that I published? > > Thanks for your help in advance. > > best regards, > nickolay kolev > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 4, 2013, 9:31am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/4 "2013-07-04T09:31:15Z")
These are not errors but only info. When you created the river, a new document type was created (rabbit\_river2). That's what these logs are saying. If you want to turn debug on, modify logging.yml file and set river.rabbitmq: trace You should see some details. -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 4 juil. 2013 à 11:26, Николай Колев [lem00na@gmail.com](mailto:lem00na@gmail.com) a écrit : > Hi David, > > There were some errors while I was trying to figure what I have to do but now it seems that it is correct and works: > [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings > [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) > [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses [[192.168.106.11:5672]], user [guest], vhost [playground] > [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) > > Is there and option to make it more verbose? > > best regards, > nickolay kolev > > ## 04 юли 2013, четвъртък, 12:22:57 UTC+3, David Pilato написа: Anything in logs? > > David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) > @dadoonet | @elasticsearchfr | @scrutmydocs > > Le 4 juil. 2013 à 11:06, Николай Колев [lem...@gmail.com](mailto:lem...@gmail.com) a écrit : > > > Hi all, > > > > I am new to RabbitMQ and Elasticsearch. I am trying to create a setup where RabbitMQ collects messages from some sources and publish then in Elasticsearch for indexing. > > What I have done so far: > > > > Installed Elasticsearch 0.9.2 on Ubuntu 12.04 > > Created index named rabbit2 with these mappings: > > { > > "settings" : { > > "number\_of\_shards" : 5 > > }, > > "mappings" : { > > "logentry" : { > > "\_all" : {"enabled" : false}, > > "Metadata" : { "type" : "string", "index" : "not\_analyzed" }, > > "Categories" : {"type" : "string", "index" : "not\_analyzed" }, > > "ExtendedProperties" : { > > "type" : "object", > > "properties" : { > > "DocumentKey": {"type" : "string", "index" : "not\_analyzed"}, > > "Message": {"type" : "string", "index" : "analyzed"}, > > "ErrorCode": {"type" : "integer", "store" : "yes" }, > > "InnerMessage": {"type" : "string", "index": "analyzed"} > > } > > }, > > "Message": {"type" : "string", "index" : "not\_analyzed"}, > > "Timestamp": {"type" : "date", "format" : "date\_time" } > > } > > } > > } > > > > Installed RabbitMQ river and configured it with this settings > > { > > "type" : "rabbitmq", > > "rabbitmq" : { > > "host" : "192.168.106.11", > > "port" : 5672, > > "user" : "guest", > > "pass" : "guest", > > "vhost" : "playground", > > "queue" : "platform.flume.queue", > > "exchange" : "platform.flume.exchange", > > "routing\_key" : "flume", > > "exchange\_declare" : false, > > "exchange\_type" : "direct", > > "exchange\_durable" : false, > > "queue\_declare" : false, > > "queue\_bind" : true, > > "queue\_durable" : true, > > "queue\_auto\_delete" : false > > }, > > "index" : { > > "name": "rabbit2", > > "type" : "logentry", > > "bulk\_size" : 100, > > "bulk\_timeout" : "10ms", > > "ordered" : false > > } > > } > > > > ES log shows no error, RabbitMQ Web UI shows that there is a consumer connected with the IP address of the ES server > > RabbitMQ exchange and queue are predefined and already existing. > > > > I have published from RabbitMQ Web UI following message: > > > > { "create": { "\_index": "rabbit2", "\_type": "logentry", "\_id": 2 }} > > {"Metadata": ["PLATFORM-PL1","10.176.130.201;","Platform.CPBC.Container.exe","Platform.CPBC.Container","5584", null,"4020","0","Error"],"Categories": ["PLATFORM-PL1.platform.cpbc.data"],"ExtendedProperties": {"DocumentKey": "tenant-alerts\_1e5d2fa7-0e7a-4f02-b91a-cf09e13e66af", "Message": "Failed to process response: Not found", "ErrorCode": 1, "InnerMessage": "Failed to process response: Not found" }, "Message": "Document was not found.", "Timestamp": "2013-06-26T18:00:07.6122425Z"} > > > > Unfortunately the message is not consumed by ES and stays as "Unacknowledged". > > > > My question what I am doing wrong? Did I created wrong configuration or missed something? > > Is there somthing wrong with the message that I published? > > > > Thanks for your help in advance. > > > > best regards, > > nickolay kolev > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![11124](https://avatars.discourse-cdn.com/v4/letter/1/b5a626/32.png) [@11124](https://discuss.elastic.co/u/11124)\ **Post date:** [July 4, 2013, 9:49am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/5 "2013-07-04T09:49:04Z")
Hi David, I have added this option and restarted ES server. Here is the log: [2013-07-04 11:23:23,772][INFO][cluster.metadata] [Man-Thing] [\_river] deleting index [2013-07-04 11:23:23,774][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] closing rabbitmq river [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses [[192.168.106.11:5672]], user [guest], vhost [playground] [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) [2013-07-04 12:38:56,592][INFO][node] [Man-Thing] {0.90.2}[5943]: stopping ... [2013-07-04 12:38:56,632][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] closing rabbitmq river [2013-07-04 12:38:56,705][INFO][node] [Man-Thing] {0.90.2}[5943]: stopped [2013-07-04 12:38:56,705][INFO][node] [Man-Thing] {0.90.2}[5943]: closing ... [2013-07-04 12:38:56,720][INFO][node] [Man-Thing] {0.90.2}[5943]: closed [2013-07-04 12:38:59,131][INFO][node] [X-23] {0.90.2}[27353]: initializing ... [2013-07-04 12:38:59,149][INFO][plugins] [X-23] loaded [river-rabbitmq], sites [bigdesk, HQ, head] [2013-07-04 12:39:03,145][INFO][node] [X-23] {0.90.2}[27353]: initialized [2013-07-04 12:39:03,145][INFO][node] [X-23] {0.90.2}[27353]: starting ... [2013-07-04 12:39:03,296][INFO][transport] [X-23] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address {inet[/ 192.168.104.134:9300]} [2013-07-04 12:39:06,402][INFO][cluster.service] [X-23] new\_master [X-23][Jwk5Jl-PR-68g4c1VA4vSQ][inet[/192.168.104.134:9300]], reason: zen-disco-join (elected\_as\_master) [2013-07-04 12:39:06,450][INFO][discovery] [X-23] elasticsearch/Jwk5Jl-PR-68g4c1VA4vSQ [2013-07-04 12:39:06,480][INFO][http] [X-23] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish\_address {inet[/ 192.168.104.134:9200]} [2013-07-04 12:39:06,481][INFO][node] [X-23] {0.90.2}[27353]: started [2013-07-04 12:39:07,638][INFO][gateway] [X-23] recovered [7] indices into cluster\_state [2013-07-04 12:39:07,713][INFO][river.rabbitmq] [X-23] [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses [[192.168.106.11:5672]], user [guest], vhost [playground] [2013-07-04 12:43:43,981][TRACE][river.rabbitmq] [X-23] [rabbitmq][rabbit\_river2] executing bulk with [0] actions [2013-07-04 12:44:30,979][TRACE][river.rabbitmq] [X-23] [rabbitmq][rabbit\_river2] executing bulk with [0] actions To me it looks like the message that I push in RabbitMQ queue is incorrect but I cannot figure out what is the problem. Any suggestions? Are any docs about the structure of bulk message – I was not able to find any Thanks best regards, Nickolay kolev 2013/7/4 David Pilato [david@pilato.fr](mailto:david@pilato.fr) > These are not errors but only info. > When you created the river, a new document type was created > (rabbit\_river2). That's what these logs are saying. > > If you want to turn debug on, modify logging.yml file and set > river.rabbitmq: trace > > You should see some details. > > -- > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 4 juil. 2013 à 11:26, Николай Колев [lem00na@gmail.com](mailto:lem00na@gmail.com) a écrit : > > Hi David, > > There were some errors while I was trying to figure what I have to do but > now it seems that it is correct and works: > [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] > [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings > > [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] > [\_river] update\_mapping [rabbit\_river2] (dynamic) > [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] > [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses > [[192.168.106.11:5672]], user [guest], vhost [playground] > [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] > [\_river] update\_mapping [rabbit\_river2] (dynamic) > > Is there and option to make it more verbose? > > best regards, > nickolay kolev > > 04 юли 2013, четвъртък, 12:22:57 UTC+3, David Pilato написа: > > > ## Anything in logs? > > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)[http://elasticsearch.com/](http://elasticsearch.com/) > > \* > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > > |_\* @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > > > Le 4 juil. 2013 à 11:06, Николай Колев [lem...@gmail.com](mailto:lem...@gmail.com) a écrit : > > > > Hi all, > > > > I am new to RabbitMQ and Elasticsearch. I am trying to create a setup > > where RabbitMQ collects messages from some sources and publish then in > > Elasticsearch for indexing. > > What I have done so far: > > > > Installed Elasticsearch 0.9.2 on Ubuntu 12.04 > > Created index named rabbit2 with these mappings: > > { > > "settings" : { > > "number\_of\_shards" : 5 > > }, > > "mappings" : { > > "logentry" : { > > "\_all" : {"enabled" : false}, > > "Metadata" : { "type" : "string", "index" : "not\_analyzed" }, > > "Categories" : {"type" : "string", "index" : "not\_analyzed" }, > > "ExtendedProperties" : { > > "type" : "object", > > "properties" : { > > "DocumentKey": {"type" : "string", "index" : "not\_analyzed"}, > > "Message": {"type" : "string", "index" : "analyzed"}, > > "ErrorCode": {"type" : "integer", "store" : "yes" }, > > "InnerMessage": {"type" : "string", "index": "analyzed"} > > } > > }, > > "Message": {"type" : "string", "index" : "not\_analyzed"}, > > "Timestamp": {"type" : "date", "format" : "date\_time" } > > } > > } > > } > > > > Installed RabbitMQ river and configured it with this settings > > { > > "type" : "rabbitmq", > > "rabbitmq" : { > > "host" : "192.168.106.11", > > "port" : 5672, > > "user" : "guest", > > "pass" : "guest", > > "vhost" : "playground", > > "queue" : "platform.flume.queue", > > "exchange" : "platform.flume.exchange", > > "routing\_key" : "flume", > > "exchange\_declare" : false, > > "exchange\_type" : "direct", > > "exchange\_durable" : false, > > "queue\_declare" : false, > > "queue\_bind" : true, > > "queue\_durable" : true, > > "queue\_auto\_delete" : false > > }, > > "index" : { > > "name": "rabbit2", > > "type" : "logentry", > > "bulk\_size" : 100, > > "bulk\_timeout" : "10ms", > > "ordered" : false > > } > > } > > > > ES log shows no error, RabbitMQ Web UI shows that there is a consumer > > connected with the IP address of the ES server > > RabbitMQ exchange and queue are predefined and already existing. > > > > I have published from RabbitMQ Web UI following message: > > > > { "create": { "\_index": "rabbit2", "\_type": "logentry", "\_id": 2 }} > > {"Metadata": ["PLATFORM-PL1","10.176.130. **201;","Platform.CPBC.** > > Container.exe","Platform.CPBC.\*\*Container","5584", > > null,"4020","0","Error"],"\*\*Categories": ["PLATFORM-PL1.platform.cpbc.\*\*data"],"ExtendedProperties": > > {"DocumentKey": "tenant-alerts\_1e5d2fa7-0e7a-**4f02-b91a-cf09e13e66af", > > "Message": "Failed to process response: Not found", "ErrorCode": 1, > > "InnerMessage": "Failed to process response: Not found" }, "Message": > > "Document was not found.", "Timestamp": "2013-06-26T18:00:07.6122425Z"**} > > > > Unfortunately the message is not consumed by ES and stays as > > "Unacknowledged". > > > > My question what I am doing wrong? Did I created wrong configuration or > > missed something? > > Is there somthing wrong with the message that I published? > > > > Thanks for your help in advance. > > > > best regards, > > nickolay kolev > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > . > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 4, 2013, 10:02am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/6 "2013-07-04T10:02:29Z")
Check as well that your last line has a carriage return. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 4 juil. 2013 à 11:49, lem0na [lem00na@gmail.com](mailto:lem00na@gmail.com) a écrit : > Hi David, > > I have added this option and restarted ES server. Here is the log: > [2013-07-04 11:23:23,772][INFO][cluster.metadata] [Man-Thing] [\_river] deleting index > [2013-07-04 11:23:23,774][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] closing rabbitmq river > [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings > [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) > [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses [[192.168.106.11:5672]], user [guest], vhost [playground] > [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) > [2013-07-04 12:38:56,592][INFO][node] [Man-Thing] {0.90.2}[5943]: stopping ... > [2013-07-04 12:38:56,632][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] closing rabbitmq river > [2013-07-04 12:38:56,705][INFO][node] [Man-Thing] {0.90.2}[5943]: stopped > [2013-07-04 12:38:56,705][INFO][node] [Man-Thing] {0.90.2}[5943]: closing ... > [2013-07-04 12:38:56,720][INFO][node] [Man-Thing] {0.90.2}[5943]: closed > [2013-07-04 12:38:59,131][INFO][node] [X-23] {0.90.2}[27353]: initializing ... > [2013-07-04 12:38:59,149][INFO][plugins] [X-23] loaded [river-rabbitmq], sites [bigdesk, HQ, head] > [2013-07-04 12:39:03,145][INFO][node] [X-23] {0.90.2}[27353]: initialized > [2013-07-04 12:39:03,145][INFO][node] [X-23] {0.90.2}[27353]: starting ... > [2013-07-04 12:39:03,296][INFO][transport] [X-23] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address {inet[/192.168.104.134:9300]} > [2013-07-04 12:39:06,402][INFO][cluster.service] [X-23] new\_master [X-23][Jwk5Jl-PR-68g4c1VA4vSQ][inet[/192.168.104.134:9300]], reason: zen-disco-join (elected\_as\_master) > [2013-07-04 12:39:06,450][INFO][discovery] [X-23] elasticsearch/Jwk5Jl-PR-68g4c1VA4vSQ > [2013-07-04 12:39:06,480][INFO][http] [X-23] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish\_address {inet[/192.168.104.134:9200]} > [2013-07-04 12:39:06,481][INFO][node] [X-23] {0.90.2}[27353]: started > [2013-07-04 12:39:07,638][INFO][gateway] [X-23] recovered [7] indices into cluster\_state > [2013-07-04 12:39:07,713][INFO][river.rabbitmq] [X-23] [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses [[192.168.106.11:5672]], user [guest], vhost [playground] > [2013-07-04 12:43:43,981][TRACE][river.rabbitmq] [X-23] [rabbitmq][rabbit\_river2] executing bulk with [0] actions > [2013-07-04 12:44:30,979][TRACE][river.rabbitmq] [X-23] [rabbitmq][rabbit\_river2] executing bulk with [0] actions > > To me it looks like the message that I push in RabbitMQ queue is incorrect but I cannot figure out what is the problem. Any suggestions? Are any docs about the structure of bulk message – I was not able to find any > > Thanks > > best regards, > Nickolay kolev > > 2013/7/4 David Pilato [david@pilato.fr](mailto:david@pilato.fr) > > > These are not errors but only info. > > When you created the river, a new document type was created (rabbit\_river2). That's what these logs are saying. > > > > If you want to turn debug on, modify logging.yml file and set river.rabbitmq: trace > > > > You should see some details. > > > > -- > > David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) > > @dadoonet | @elasticsearchfr | @scrutmydocs > > > > Le 4 juil. 2013 à 11:26, Николай Колев [lem00na@gmail.com](mailto:lem00na@gmail.com) a écrit : > > > > > Hi David, > > > > > > There were some errors while I was trying to figure what I have to do but now it seems that it is correct and works: > > > [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings > > > [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) > > > [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses [[192.168.106.11:5672]], user [guest], vhost [playground] > > > [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] [\_river] update\_mapping [rabbit\_river2] (dynamic) > > > > > > Is there and option to make it more verbose? > > > > > > best regards, > > > nickolay kolev > > > > > > 04 юли 2013, четвъртък, 12:22:57 UTC+3, David Pilato написа: > > > > > > > ## Anything in logs? > > > > > > > > David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) > > > > @dadoonet | @elasticsearchfr | @scrutmydocs > > > > > > > > Le 4 juil. 2013 à 11:06, Николай Колев [lem...@gmail.com](mailto:lem...@gmail.com) a écrit : > > > > > > > > > Hi all, > > > > > > > > > > I am new to RabbitMQ and Elasticsearch. I am trying to create a setup where RabbitMQ collects messages from some sources and publish then in Elasticsearch for indexing. > > > > > What I have done so far: > > > > > > > > > > Installed Elasticsearch 0.9.2 on Ubuntu 12.04 > > > > > Created index named rabbit2 with these mappings: > > > > > { > > > > > "settings" : { > > > > > "number\_of\_shards" : 5 > > > > > }, > > > > > "mappings" : { > > > > > "logentry" : { > > > > > "\_all" : {"enabled" : false}, > > > > > "Metadata" : { "type" : "string", "index" : "not\_analyzed" }, > > > > > "Categories" : {"type" : "string", "index" : "not\_analyzed" }, > > > > > "ExtendedProperties" : { > > > > > "type" : "object", > > > > > "properties" : { > > > > > "DocumentKey": {"type" : "string", "index" : "not\_analyzed"}, > > > > > "Message": {"type" : "string", "index" : "analyzed"}, > > > > > "ErrorCode": {"type" : "integer", "store" : "yes" }, > > > > > "InnerMessage": {"type" : "string", "index": "analyzed"} > > > > > } > > > > > }, > > > > > "Message": {"type" : "string", "index" : "not\_analyzed"}, > > > > > "Timestamp": {"type" : "date", "format" : "date\_time" } > > > > > } > > > > > } > > > > > } > > > > > > > > > > Installed RabbitMQ river and configured it with this settings > > > > > { > > > > > "type" : "rabbitmq", > > > > > "rabbitmq" : { > > > > > "host" : "192.168.106.11", > > > > > "port" : 5672, > > > > > "user" : "guest", > > > > > "pass" : "guest", > > > > > "vhost" : "playground", > > > > > "queue" : "platform.flume.queue", > > > > > "exchange" : "platform.flume.exchange", > > > > > "routing\_key" : "flume", > > > > > "exchange\_declare" : false, > > > > > "exchange\_type" : "direct", > > > > > "exchange\_durable" : false, > > > > > "queue\_declare" : false, > > > > > "queue\_bind" : true, > > > > > "queue\_durable" : true, > > > > > "queue\_auto\_delete" : false > > > > > }, > > > > > "index" : { > > > > > "name": "rabbit2", > > > > > "type" : "logentry", > > > > > "bulk\_size" : 100, > > > > > "bulk\_timeout" : "10ms", > > > > > "ordered" : false > > > > > } > > > > > } > > > > > > > > > > ES log shows no error, RabbitMQ Web UI shows that there is a consumer connected with the IP address of the ES server > > > > > RabbitMQ exchange and queue are predefined and already existing. > > > > > > > > > > I have published from RabbitMQ Web UI following message: > > > > > > > > > > { "create": { "\_index": "rabbit2", "\_type": "logentry", "\_id": 2 }} > > > > > {"Metadata": ["PLATFORM-PL1","10.176.130.201;","Platform.CPBC.Container.exe","Platform.CPBC.Container","5584", null,"4020","0","Error"],"Categories": ["PLATFORM-PL1.platform.cpbc.data"],"ExtendedProperties": {"DocumentKey": "tenant-alerts\_1e5d2fa7-0e7a-4f02-b91a-cf09e13e66af", "Message": "Failed to process response: Not found", "ErrorCode": 1, "InnerMessage": "Failed to process response: Not found" }, "Message": "Document was not found.", "Timestamp": "2013-06-26T18:00:07.6122425Z"} > > > > > > > > > > Unfortunately the message is not consumed by ES and stays as "Unacknowledged". > > > > > > > > > > My question what I am doing wrong? Did I created wrong configuration or missed something? > > > > > Is there somthing wrong with the message that I published? > > > > > > > > > > Thanks for your help in advance. > > > > > > > > > > best regards, > > > > > nickolay kolev > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![11124](https://avatars.discourse-cdn.com/v4/letter/1/b5a626/32.png) [@11124](https://discuss.elastic.co/u/11124)\ **Post date:** [July 4, 2013, 10:14am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/7 "2013-07-04T10:14:31Z")
Hi David, Missing carriage return was the problem. Thanks for your patience and support. best regards, nickolay kolev 04 юли 2013, четвъртък, 13:02:29 UTC+3, David Pilato написа: > Check as well that your last line has a carriage return. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 4 juil. 2013 à 11:49, lem0na \<[lem...@gmail.com](mailto:lem...@gmail.com) \\> a écrit : > > Hi David, > > I have added this option and restarted ES server. Here is the log: > [2013-07-04 11:23:23,772][INFO][cluster.metadata] [Man-Thing] > [\_river] deleting index > [2013-07-04 11:23:23,774][INFO][river.rabbitmq] [Man-Thing] > [rabbitmq][rabbit\_river2] closing rabbitmq river > [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] > [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings > > [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] > [\_river] update\_mapping [rabbit\_river2] (dynamic) > [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] > [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses > [[192.168.106.11:5672]], user [guest], vhost [playground] > [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] > [\_river] update\_mapping [rabbit\_river2] (dynamic) > [2013-07-04 12:38:56,592][INFO][node] [Man-Thing] > {0.90.2}[5943]: stopping ... > [2013-07-04 12:38:56,632][INFO][river.rabbitmq] [Man-Thing] > [rabbitmq][rabbit\_river2] closing rabbitmq river > [2013-07-04 12:38:56,705][INFO][node] [Man-Thing] > {0.90.2}[5943]: stopped > [2013-07-04 12:38:56,705][INFO][node] [Man-Thing] > {0.90.2}[5943]: closing ... > [2013-07-04 12:38:56,720][INFO][node] [Man-Thing] > {0.90.2}[5943]: closed > [2013-07-04 12:38:59,131][INFO][node] [X-23] > {0.90.2}[27353]: initializing ... > [2013-07-04 12:38:59,149][INFO][plugins] [X-23] loaded > [river-rabbitmq], sites [bigdesk, HQ, head] > [2013-07-04 12:39:03,145][INFO][node] [X-23] > {0.90.2}[27353]: initialized > [2013-07-04 12:39:03,145][INFO][node] [X-23] > {0.90.2}[27353]: starting ... > [2013-07-04 12:39:03,296][INFO][transport] [X-23] > bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address {inet[/ > 192.168.104.134:9300]} > [2013-07-04 12:39:06,402][INFO][cluster.service] [X-23] > new\_master [X-23][Jwk5Jl-PR-68g4c1VA4vSQ][inet[/192.168.104.134:9300]], > reason: zen-disco-join (elected\_as\_master) > [2013-07-04 12:39:06,450][INFO][discovery] [X-23] > elasticsearch/Jwk5Jl-PR-68g4c1VA4vSQ > [2013-07-04 12:39:06,480][INFO][http] [X-23] > bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish\_address {inet[/ > 192.168.104.134:9200]} > [2013-07-04 12:39:06,481][INFO][node] [X-23] > {0.90.2}[27353]: started > [2013-07-04 12:39:07,638][INFO][gateway] [X-23] > recovered [7] indices into cluster\_state > [2013-07-04 12:39:07,713][INFO][river.rabbitmq] [X-23] > [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses > [[192.168.106.11:5672]], user [guest], vhost [playground] > [2013-07-04 12:43:43,981][TRACE][river.rabbitmq] [X-23] > [rabbitmq][rabbit\_river2] executing bulk with [0] actions > [2013-07-04 12:44:30,979][TRACE][river.rabbitmq] [X-23] > [rabbitmq][rabbit\_river2] executing bulk with [0] actions > > To me it looks like the message that I push in RabbitMQ queue is incorrect > but I cannot figure out what is the problem. Any suggestions? Are any > docs about the structure of bulk message – I was not able to find any > > Thanks > > best regards, > Nickolay kolev > > 2013/7/4 David Pilato \<[da...@pilato.fr](mailto:da...@pilato.fr) \\> > > > These are not errors but only info. > > When you created the river, a new document type was created > > (rabbit\_river2). That's what these logs are saying. > > > > If you want to turn debug on, modify logging.yml file and set > > river.rabbitmq: trace > > > > You should see some details. > > > > -- > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > > > Le 4 juil. 2013 à 11:26, Николай Колев \<[lem...@gmail.com](mailto:lem...@gmail.com) \\> > > a écrit : > > > > Hi David, > > > > There were some errors while I was trying to figure what I have to do but > > now it seems that it is correct and works: > > [2013-07-04 11:23:40,261][INFO][cluster.metadata] [Man-Thing] > > [\_river] creating index, cause [auto(index api)], shards [1]/[1], mappings > > > > [2013-07-04 11:23:40,305][INFO][cluster.metadata] [Man-Thing] > > [\_river] update\_mapping [rabbit\_river2] (dynamic) > > [2013-07-04 11:23:40,308][INFO][river.rabbitmq] [Man-Thing] > > [rabbitmq][rabbit\_river2] creating rabbitmq river, addresses > > [[192.168.106.11:5672]], user [guest], vhost [playground] > > [2013-07-04 11:23:40,338][INFO][cluster.metadata] [Man-Thing] > > [\_river] update\_mapping [rabbit\_river2] (dynamic) > > > > Is there and option to make it more verbose? > > > > best regards, > > nickolay kolev > > > > 04 юли 2013, четвъртък, 12:22:57 UTC+3, David Pilato написа: > > > > > ## Anything in logs? > > > > > > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)[http://elasticsearch.com/](http://elasticsearch.com/) > > > \* > > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > > > |_\* @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > > > > > Le 4 juil. 2013 à 11:06, Николай Колев [lem...@gmail.com](mailto:lem...@gmail.com) a écrit : > > > > > > Hi all, > > > > > > I am new to RabbitMQ and Elasticsearch. I am trying to create a setup > > > where RabbitMQ collects messages from some sources and publish then in > > > Elasticsearch for indexing. > > > What I have done so far: > > > > > > Installed Elasticsearch 0.9.2 on Ubuntu 12.04 > > > Created index named rabbit2 with these mappings: > > > { > > > "settings" : { > > > "number\_of\_shards" : 5 > > > }, > > > "mappings" : { > > > "logentry" : { > > > "\_all" : {"enabled" : false}, > > > "Metadata" : { "type" : "string", "index" : "not\_analyzed" }, > > > "Categories" : {"type" : "string", "index" : "not\_analyzed" }, > > > "ExtendedProperties" : { > > > "type" : "object", > > > "properties" : { > > > "DocumentKey": {"type" : "string", "index" : "not\_analyzed"}, > > > "Message": {"type" : "string", "index" : "analyzed"}, > > > "ErrorCode": {"type" : "integer", "store" : "yes" }, > > > "InnerMessage": {"type" : "string", "index": "analyzed"} > > > } > > > }, > > > "Message": {"type" : "string", "index" : "not\_analyzed"}, > > > "Timestamp": {"type" : "date", "format" : "date\_time" } > > > } > > > } > > > } > > > > > > Installed RabbitMQ river and configured it with this settings > > > { > > > "type" : "rabbitmq", > > > "rabbitmq" : { > > > "host" : "192.168.106.11", > > > "port" : 5672, > > > "user" : "guest", > > > "pass" : "guest", > > > "vhost" : "playground", > > > "queue" : "platform.flume.queue", > > > "exchange" : "platform.flume.exchange", > > > "routing\_key" : "flume", > > > "exchange\_declare" : false, > > > "exchange\_type" : "direct", > > > "exchange\_durable" : false, > > > "queue\_declare" : false, > > > "queue\_bind" : true, > > > "queue\_durable" : true, > > > "queue\_auto\_delete" : false > > > }, > > > "index" : { > > > "name": "rabbit2", > > > "type" : "logentry", > > > "bulk\_size" : 100, > > > "bulk\_timeout" : "10ms", > > > "ordered" : false > > > } > > > } > > > > > > ES log shows no error, RabbitMQ Web UI shows that there is a consumer > > > connected with the IP address of the ES server > > > RabbitMQ exchange and queue are predefined and already existing. > > > > > > I have published from RabbitMQ Web UI following message: > > > > > > { "create": { "\_index": "rabbit2", "\_type": "logentry", "\_id": 2 }} > > > {"Metadata": ["PLATFORM-PL1","10.176.130. **201;","Platform.CPBC.** > > > Container.exe","Platform.CPBC.\*\*Container","5584", > > > null,"4020","0","Error"],"\*\*Categories": ["PLATFORM-PL1.platform.cpbc.\*\*data"],"ExtendedProperties": > > > {"DocumentKey": "tenant-alerts\_1e5d2fa7-0e7a-**4f02-b91a-cf09e13e66af", > > > "Message": "Failed to process response: Not found", "ErrorCode": 1, > > > "InnerMessage": "Failed to process response: Not found" }, "Message": > > > "Document was not found.", "Timestamp": "2013-06-26T18:00:07.6122425Z"** > > > } > > > > > > Unfortunately the message is not consumed by ES and stays as > > > "Unacknowledged". > > > > > > My question what I am doing wrong? Did I created wrong configuration or > > > missed something? > > > Is there somthing wrong with the message that I published? > > > > > > Thanks for your help in advance. > > > > > > best regards, > > > nickolay kolev > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > . > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![yuheng](https://avatars.discourse-cdn.com/v4/letter/y/b5ac83/32.png) [@yuheng](https://discuss.elastic.co/u/yuheng)\ **Post date:** [June 10, 2014, 3:13pm UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/8 "2014-06-10T15:13:03Z")
Hi All, I am using the rabbitmq-river plugin for elasticsearch. My configuration for the river is as follows: curl -XPUT 'localhost:9200/\_river/rabbit\_river/\_meta' -d '{ "type" : "rabbitmq", "rabbitmq" : { "host" : "lbha1.ir.clemson.edu", "port" : 5672, "user" : "guest", "pass" : "guest", "vhost" : "/", "queue" : "diagnostics\_es", "exchange" : "ir.diagnostics", "routing\_key" : "ir.diagnostics._._", "exchange\_declare" : true, "exchange\_type" : "topic", "exchange\_durable" : true, "queue\_declare" : true, "queue\_bind" : true, "queue\_durable" : true, "queue\_auto\_delete" : false, "heartbeat" : "30m", "nack\_errors" : true }, "index" : { "bulk\_size" : 100, "bulk\_timeout" : "10ms", "ordered" : false, "replication" : "default" } }' However, the logs of elasticsearch keeps give me the following message: [2014-06-10 11:12:33,354][TRACE][river.rabbitmq] [Madame Web] [rabbitmq][rabbit\_river] executing bulk with [0] actions Do you have any idea why this happens? Thanks! best, Yuheng -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/da7442f8-7eef-4044-b4c8-a78110f98186%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/da7442f8-7eef-4044-b4c8-a78110f98186%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:23am UTC](https://discuss.elastic.co/t/elasticsearch-and-rabbitmq-as-data-source/12659/9 "2017-07-06T01:23:34Z")