# Kafka River Plugin reference problem **URL:** \ **Category:** Elasticsearch\ **Created:** [March 13, 2013, 8:34am UTC](https://discuss.elastic.co/t/kafka-river-plugin-reference-problem/11128 "2013-03-13T08:34:39Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![xiaoliang\_tian](https://avatars.discourse-cdn.com/v4/letter/x/a9a28c/32.png) [@xiaoliang\_tian](https://discuss.elastic.co/u/xiaoliang_tian)\ **Post date:** [March 13, 2013, 8:34am UTC](https://discuss.elastic.co/t/kafka-river-plugin-reference-problem/11128/1 "2013-03-13T08:34:39Z")
Hi,I am writing a kafka river plugin here is the thing: for example There is 5 partition in kafka So I think the river plugin should only read 1 partition ,like that Map\ topicCountMap = new ConcurrentHashMap\(); topicCountMap.put(topic, new Integer(1)); Map\\> consumerMap = consumer .createMessageStreams(topicCountMap); And the elastic search cluster should be consist of not more than 5 node,is that right? thanks very much. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 13, 2013, 8:44am UTC](https://discuss.elastic.co/t/kafka-river-plugin-reference-problem/11128/2 "2013-03-13T08:44:39Z")
I'm afraid I did not understand but let me say something about rivers. Whatever the number of nodes you have, one single river will only run on a single node. So I don't see any relative numbers between the amount of kafka partitions and the number of nodes. Does it help? -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 13 mars 2013 à 09:34, 田晓亮 [xiaoliang.tian@gmail.com](mailto:xiaoliang.tian@gmail.com) a écrit : > Hi,I am writing a kafka river plugin > > here is the thing: > for example > There is 5 partition in kafka > > So I think the river plugin should only read 1 partition ,like that > > ``` > Map topicCountMap = new ConcurrentHashMap(); > topicCountMap.put(topic, new Integer(1)); > > Map>> consumerMap = consumer > .createMessageStreams(topicCountMap); > > ``` > > And the Elasticsearch cluster should be consist of not more than 5 node,is that right? > > thanks very much. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![xiaoliang\_tian](https://avatars.discourse-cdn.com/v4/letter/x/a9a28c/32.png) [@xiaoliang\_tian](https://discuss.elastic.co/u/xiaoliang_tian)\ **Post date:** [March 13, 2013, 8:57am UTC](https://discuss.elastic.co/t/kafka-river-plugin-reference-problem/11128/3 "2013-03-13T08:57:22Z")
Thanks,u said u don't understand, mean u don't understand my word or kafka? according to your word,I think If I use a kafka river in es,The es nodes number must be not more than the number of kafka partition. thanks again 2013/3/13 David Pilato [david@pilato.fr](mailto:david@pilato.fr) > I'm afraid I did not understand but let me say something about rivers. > > Whatever the number of nodes you have, one single river will only run on a > single node. > So I don't see any relative numbers between the amount of kafka partitions > and the number of nodes. > > Does it help? > > -- > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 13 mars 2013 à 09:34, 田晓亮 [xiaoliang.tian@gmail.com](mailto:xiaoliang.tian@gmail.com) a écrit : > > Hi,I am writing a kafka river plugin > > here is the thing: > for example > There is 5 partition in kafka > > So I think the river plugin should only read 1 partition ,like that > > Map\ topicCountMap = new ConcurrentHashMap\ Integer\>(); > topicCountMap.put(topic, new Integer(1)); > > Map\\> consumerMap = consumer > .createMessageStreams(topicCountMap); > > And the Elasticsearch cluster should be consist of not more than 5 > node,is that right? > > thanks very much. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/R2tpcHxotX8/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/R2tpcHxotX8/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:46am UTC](https://discuss.elastic.co/t/kafka-river-plugin-reference-problem/11128/4 "2017-07-06T02:46:50Z")