Is elasticsearch-hadoop could write to 3 nodes at the same time?

Elastic Stack Elasticsearch
1

I want write documents to some nodes at the same, but I cannot find how
to add node for more than one.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hey,

can you maybe write a little bit more than only a subject line like you do
in most of your questions. Explaining what are trying to do might help
people to understand the issue a bit better and maybe allows people to
propose workarounds if the exact thing you are asking for cannot be
achieved. This type of yes/no questions are usually a little short handed
in complex products like search engines...

So, why do want to write to three nodes at one time? Do you want to write
to three different clusters? Is this about data security? What is your goal?

Thanks!

--Alex

On Wed, Jul 10, 2013 at 4:15 AM, dancer chuanhua.deng@gmail.com wrote:

I want write documents to some nodes at the same, but I cannot find how
to add node for more than one.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

hi, thanks for your reply.
I just want to import to one cluster, as I see, it may have more than one
map job to import from source, but I cannot find how to balance that data
to more than one node.

在 2013年7月10日星期三UTC+8下午4时54分47秒，Alexander Reelsen写道：

Hey,

can you maybe write a little bit more than only a subject line like you do
in most of your questions. Explaining what are trying to do might help
people to understand the issue a bit better and maybe allows people to
propose workarounds if the exact thing you are asking for cannot be
achieved. This type of yes/no questions are usually a little short handed
in complex products like search engines...

So, why do want to write to three nodes at one time? Do you want to write
to three different clusters? Is this about data security? What is your goal?

Thanks!

--Alex

On Wed, Jul 10, 2013 at 4:15 AM, dancer <chuanh...@gmail.com <javascript:>

wrote:

I want write documents to some nodes at the same, but I cannot find how
to add node for more than one.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hi,

I don't understand what are you trying to do exactly and what do you mean by cluster (is it Hadoop or ES?) but I'll try
to answer with what's currently available today in elasticsearch-hadoop:

  1. reading happens across the ES cluster - that is when you access data, we automatically split the reading across the
    available ES shards.
  2. writing happens to only one node currently (mainly because we are tied to whether the input source which is outside
    ES is splittable or not and whether those can be mapped onto the ES shards properly). Also from the few benchmarks ran,
    one ES node can index data quite fast and it can handle multiple Hadoop tasks concurrently without any degraded performance.

These being said, we plan to optimize this in the future but it's relatively low priority at some point (due to the
reasons above).

On 10/07/2013 12:32 PM, dancer wrote:

hi, thanks for your reply.
I just want to import to one cluster, as I see, it may have more than one map job to import from source, but I cannot
find how to balance that data to more than one node.

在 2013年7月10日星期三UTC+8下午4时54分47秒，Alexander Reelsen写道：

Hey,

can you maybe write a little bit more than only a subject line like you do in most of your questions. Explaining
what are trying to do might help people to understand the issue a bit better and maybe allows people to propose
workarounds if the exact thing you are asking for cannot be achieved. This type of yes/no questions are usually a
little short handed in complex products like search engines...

So, why do want to write to three nodes at one time? Do you want to write to three different clusters? Is this about
data security? What is your goal?

Thanks!


--Alex


On Wed, Jul 10, 2013 at 4:15 AM, dancer <chuanh...@gmail.com <javascript:>> wrote:

      I want write documents to some nodes  at the same, but I cannot find how to add node for more than one.

    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearc...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/groups/opt_out <https://groups.google.com/groups/opt_out>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

that is what I finding, thanks you.
my boss want writing to multi ES nodes using hadoop, but I find that plugin
could write to only one node.

在 2013年7月10日星期三UTC+8下午7时11分09秒，Costin Leau写道：

Hi,

I don't understand what are you trying to do exactly and what do you mean
by cluster (is it Hadoop or ES?) but I'll try
to answer with what's currently available today in elasticsearch-hadoop:

  1. reading happens across the ES cluster - that is when you access data,
    we automatically split the reading across the
    available ES shards.
  2. writing happens to only one node currently (mainly because we are tied
    to whether the input source which is outside
    ES is splittable or not and whether those can be mapped onto the ES shards
    properly). Also from the few benchmarks ran,
    one ES node can index data quite fast and it can handle multiple Hadoop
    tasks concurrently without any degraded performance.

These being said, we plan to optimize this in the future but it's
relatively low priority at some point (due to the
reasons above).

On 10/07/2013 12:32 PM, dancer wrote:

hi, thanks for your reply.
I just want to import to one cluster, as I see, it may have more than
one map job to import from source, but I cannot
find how to balance that data to more than one node.

在 2013年7月10日星期三UTC+8下午4时54分47秒，Alexander Reelsen写道：

Hey, 

can you maybe write a little bit more than only a subject line like

you do in most of your questions. Explaining

what are trying to do might help people to understand the issue a

bit better and maybe allows people to propose

workarounds if the exact thing you are asking for cannot be

achieved. This type of yes/no questions are usually a

little short handed in complex products like search engines... 

So, why do want to write to three nodes at one time? Do you want to

write to three different clusters? Is this about

data security? What is your goal? 

Thanks! 


--Alex 


On Wed, Jul 10, 2013 at 4:15 AM, dancer <chuanh...@gmail.com<javascript:>> wrote: 

      I want write documents to some nodes  at the same, but I

cannot find how to add node for more than one.

    -- 
    You received this message because you are subscribed to the

Google Groups "elasticsearch" group.

    To unsubscribe from this group and stop receiving emails from

it, send an email to

    elasticsearc...@googlegroups.com <javascript:>. 
    For more options, visit https://groups.google.com/groups/opt_out<

https://groups.google.com/groups/opt_out>.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.