Replication single threaded?


(Alexis Okuwa) #1

I started to test adding and removing nodes from my cluster and testing the
performance but I am seeing a lot of the time while the inital data sync is
single threaded. Most of the time i see 1 core at 100% and every now and
then the other cores are getting some usage but its clear that 1 core is
working 3 to 4 times harder then the other cores?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #2

There is no single threaded data sync. What you see is index recovery,
downloading shards from the gateway. You can modify the recovery settings
in configuration to minimize the recovery phases, by default, recovery is
placed in the background so it does not disturb ongoing saerch/index so
much.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexis Okuwa) #3

is the recovery also occured when its not a recovery like when its a clean
node?

On Sat, Sep 14, 2013 at 5:12 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

There is no single threaded data sync. What you see is index recovery,
downloading shards from the gateway. You can modify the recovery settings
in configuration to minimize the recovery phases, by default, recovery is
placed in the background so it does not disturb ongoing saerch/index so
much.

Jörg

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/NoTTQoKAvpE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Enjoy,
Alexis Okuwa
WojonsTech
424.835.1223

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #4

Recovery is not per node, but per index. If a node starts up, it receives
messages what shards it shall house. Then, these shards are either
activated if locally present, or the shard is received from another node.
If an index is complete over all shards, the recovery of the index is
complete.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #5