# Is elasticsearch-hadoop could write to 3 nodes at the same time? **URL:** \ **Category:** Elasticsearch\ **Created:** [July 10, 2013, 2:15am UTC](https://discuss.elastic.co/t/is-elasticsearch-hadoop-could-write-to-3-nodes-at-the-same-time/12732 "2013-07-10T02:15:16Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![dancer](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@dancer](https://discuss.elastic.co/u/dancer)\ **Post date:** [July 10, 2013, 2:15am UTC](https://discuss.elastic.co/t/is-elasticsearch-hadoop-could-write-to-3-nodes-at-the-same-time/12732/1 "2013-07-10T02:15:16Z")
I want write documents to some nodes at the same, but I cannot find how to add node for more than one. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [July 10, 2013, 8:54am UTC](https://discuss.elastic.co/t/is-elasticsearch-hadoop-could-write-to-3-nodes-at-the-same-time/12732/2 "2013-07-10T08:54:47Z")
Hey, can you maybe write a little bit more than only a subject line like you do in most of your questions. Explaining what are trying to do might help people to understand the issue a bit better and maybe allows people to propose workarounds if the exact thing you are asking for cannot be achieved. This type of yes/no questions are usually a little short handed in complex products like search engines... So, why do want to write to three nodes at one time? Do you want to write to three different clusters? Is this about data security? What is your goal? Thanks! --Alex On Wed, Jul 10, 2013 at 4:15 AM, dancer [chuanhua.deng@gmail.com](mailto:chuanhua.deng@gmail.com) wrote: > I want write documents to some nodes at the same, but I cannot find how > to add node for more than one. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dancer](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@dancer](https://discuss.elastic.co/u/dancer)\ **Post date:** [July 10, 2013, 9:32am UTC](https://discuss.elastic.co/t/is-elasticsearch-hadoop-could-write-to-3-nodes-at-the-same-time/12732/3 "2013-07-10T09:32:10Z")
hi, thanks for your reply. I just want to import to one cluster, as I see, it may have more than one map job to import from source, but I cannot find how to balance that data to more than one node. 在 2013年7月10日星期三UTC+8下午4时54分47秒，Alexander Reelsen写道： > Hey, > > can you maybe write a little bit more than only a subject line like you do > in most of your questions. Explaining what are trying to do might help > people to understand the issue a bit better and maybe allows people to > propose workarounds if the exact thing you are asking for cannot be > achieved. This type of yes/no questions are usually a little short handed > in complex products like search engines... > > So, why do want to write to three nodes at one time? Do you want to write > to three different clusters? Is this about data security? What is your goal? > > Thanks! > > --Alex > > On Wed, Jul 10, 2013 at 4:15 AM, dancer \<[chuanh...@gmail.com](mailto:chuanh...@gmail.com) \ > > > wrote: > > > I want write documents to some nodes at the same, but I cannot find how > > to add node for more than one. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [July 10, 2013, 11:11am UTC](https://discuss.elastic.co/t/is-elasticsearch-hadoop-could-write-to-3-nodes-at-the-same-time/12732/4 "2013-07-10T11:11:09Z")
Hi, I don't understand what are you trying to do exactly and what do you mean by cluster (is it Hadoop or ES?) but I'll try to answer with what's currently available today in elasticsearch-hadoop: 1. reading happens across the ES cluster - that is when you access data, we automatically split the reading across the available ES shards. 2. writing happens to only one node currently (mainly because we are tied to whether the input source which is outside ES is splittable or not and whether those can be mapped onto the ES shards properly). Also from the few benchmarks ran, one ES node can index data quite fast and it can handle multiple Hadoop tasks concurrently without any degraded performance. These being said, we plan to optimize this in the future but it's relatively low priority at some point (due to the reasons above). On 10/07/2013 12:32 PM, dancer wrote: > hi, thanks for your reply. > I just want to import to one cluster, as I see, it may have more than one map job to import from source, but I cannot > find how to balance that data to more than one node. > > 在 2013年7月10日星期三UTC+8下午4时54分47秒，Alexander Reelsen写道： > > ``` > Hey, > > can you maybe write a little bit more than only a subject line like you do in most of your questions. Explaining > what are trying to do might help people to understand the issue a bit better and maybe allows people to propose > workarounds if the exact thing you are asking for cannot be achieved. This type of yes/no questions are usually a > little short handed in complex products like search engines... > > So, why do want to write to three nodes at one time? Do you want to write to three different clusters? Is this about > data security? What is your goal? > > Thanks! > > --Alex > > On Wed, Jul 10, 2013 at 4:15 AM, dancer > wrote: > > I want write documents to some nodes at the same, but I cannot find how to add node for more than one. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > elasticsearc...@googlegroups.com . > For more options, visit https://groups.google.com/groups/opt_out . > > ``` > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dancer](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@dancer](https://discuss.elastic.co/u/dancer)\ **Post date:** [July 13, 2013, 1:29am UTC](https://discuss.elastic.co/t/is-elasticsearch-hadoop-could-write-to-3-nodes-at-the-same-time/12732/5 "2013-07-13T01:29:02Z")
that is what I finding, thanks you. my boss want writing to multi ES nodes using hadoop, but I find that plugin could write to only one node. 在 2013年7月10日星期三UTC+8下午7时11分09秒，Costin Leau写道： > Hi, > > I don't understand what are you trying to do exactly and what do you mean > by cluster (is it Hadoop or ES?) but I'll try > to answer with what's currently available today in elasticsearch-hadoop: > > 1. reading happens across the ES cluster - that is when you access data, > we automatically split the reading across the > available ES shards. > 2. writing happens to only one node currently (mainly because we are tied > to whether the input source which is outside > ES is splittable or not and whether those can be mapped onto the ES shards > properly). Also from the few benchmarks ran, > one ES node can index data quite fast and it can handle multiple Hadoop > tasks concurrently without any degraded performance. > > These being said, we plan to optimize this in the future but it's > relatively low priority at some point (due to the > reasons above). > > On 10/07/2013 12:32 PM, dancer wrote: > > > hi, thanks for your reply. > > I just want to import to one cluster, as I see, it may have more than > > one map job to import from source, but I cannot > > find how to balance that data to more than one node. > > > > 在 2013年7月10日星期三UTC+8下午4时54分47秒，Alexander Reelsen写道： > > > > ``` > > Hey, > > > > can you maybe write a little bit more than only a subject line like > > > > ``` > > you do in most of your questions. Explaining > > > ``` > > what are trying to do might help people to understand the issue a > > > > ``` > > bit better and maybe allows people to propose > > > ``` > > workarounds if the exact thing you are asking for cannot be > > > > ``` > > achieved. This type of yes/no questions are usually a > > > ``` > > little short handed in complex products like search engines... > > > > So, why do want to write to three nodes at one time? Do you want to > > > > ``` > > write to three different clusters? Is this about > > > ``` > > data security? What is your goal? > > > > Thanks! > > > > --Alex > > > > On Wed, Jul 10, 2013 at 4:15 AM, dancer > wrote: > > > > I want write documents to some nodes at the same, but I > > > > ``` > > cannot find how to add node for more than one. > > > ``` > > -- > > You received this message because you are subscribed to the > > > > ``` > > Google Groups "elasticsearch" group. > > > ``` > > To unsubscribe from this group and stop receiving emails from > > > > ``` > > it, send an email to > > > ``` > > elasticsearc...@googlegroups.com . > > For more options, visit https://groups.google.com/groups/opt_out< > > > > ``` > > [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out)\>. > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Costin -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:26am UTC](https://discuss.elastic.co/t/is-elasticsearch-hadoop-could-write-to-3-nodes-at-the-same-time/12732/6 "2017-07-06T02:26:41Z")