# Network latency while adding new replica **URL:** \ **Category:** Elasticsearch\ **Created:** [October 5, 2011, 11:37am UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521 "2011-10-05T11:37:50Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Tarun\_Jangra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_jangra/32/1426_2.png) [@Tarun\_Jangra](https://discuss.elastic.co/u/Tarun_Jangra)\ **Post date:** [October 5, 2011, 11:37am UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521/1 "2011-10-05T11:37:50Z")
I am confused with this. Elastic search is following Push replication algorithm instead of having "Pull replication algorithm". Where chunk of data are suppose to be pushed to replicas. if i have say 5 nodes, And i get a situation where i need to add more nodes to increase resources. Than obviously Shards are suppose to be replicated on all new nodes. But if shards have documents in millions than what happened with the data while coping shard to new replicas. Because system is available during this copying process. How it will manage to have updates in shard happened during this copying process? And time of availability of new replicas is actually depend over size of shard being copied? So could it vary accordingly? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 6, 2011, 7:10pm UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521/2 "2011-10-06T19:10:53Z")
Yes, the time it will take to _move_ shards around depends on the size of them. It allows for indexing operation to still occur because it uses a transaction log, which is used to make sure that the copy process can take place while indexing is still happening (through replaying it against hte copy when needed). On Wed, Oct 5, 2011 at 1:37 PM, tarun.jangra [tarun@izap.in](mailto:tarun@izap.in) wrote: > I am confused with this. Elastic search is following Push replication > algorithm instead of having "Pull replication algorithm". Where chunk of > data are suppose to be pushed to replicas. if i have say 5 nodes, And i get > a situation where i need to add more nodes to increase resources. Than > obviously Shards are suppose to be replicated on all new nodes. But if > shards have documents in millions than what happened with the data while > coping shard to new replicas. Because system is available during this > copying process. How it will manage to have updates in shard happened > during > this copying process? And time of availability of new replicas is actually > depend over size of shard being copied? So could it vary accordingly? > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Network-latency-while-adding-new-replica-tp3396237p3396237.html](http://elasticsearch-users.115913.n3.nabble.com/Network-latency-while-adding-new-replica-tp3396237p3396237.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![Tarun\_Jangra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_jangra/32/1426_2.png) [@Tarun\_Jangra](https://discuss.elastic.co/u/Tarun_Jangra)\ **Post date:** [October 6, 2011, 8:23pm UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521/3 "2011-10-06T20:23:12Z")
it means there is no chance of data loss unless whole cluster is down. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 6, 2011, 8:32pm UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521/4 "2011-10-06T20:32:45Z")
Thats a different question then moving shards around, and you won't loose data even if the whole cluster is down as long as you bring it back up using the same data location. On Thu, Oct 6, 2011 at 10:23 PM, Tarun [tarun@izap.in](mailto:tarun@izap.in) wrote: > it means there is no chance of data loss unless whole cluster is down. > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Network-latency-while-adding-new-replica-tp3396237p3400951.html](http://elasticsearch-users.115913.n3.nabble.com/Network-latency-while-adding-new-replica-tp3396237p3400951.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![Tarun\_Jangra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_jangra/32/1426_2.png) [@Tarun\_Jangra](https://discuss.elastic.co/u/Tarun_Jangra)\ **Post date:** [October 6, 2011, 8:45pm UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521/5 "2011-10-06T20:45:38Z")
hi kimchy, thanks for your prompt reply. suppose i need highly available system for 10 million documents. than what is the best combinations of shards and replicas over amazon cloud and why? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 8, 2011, 6:36pm UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521/6 "2011-10-08T18:36:39Z")
The number of shards really depends on the document structure, size of it, number of fields... . You will need to test a bit. As for number of replicas, they control how highly available you want it to be. With number\_of\_replicas set to 1, you will have two copies of your data. On Thu, Oct 6, 2011 at 10:45 PM, Tarun [tarun@izap.in](mailto:tarun@izap.in) wrote: > hi kimchy, > thanks for your prompt reply. suppose i need highly available system for 10 > million documents. than what is the best combinations of shards and > replicas > over amazon cloud and why? > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Network-latency-while-adding-new-replica-tp3396237p3401023.html](http://elasticsearch-users.115913.n3.nabble.com/Network-latency-while-adding-new-replica-tp3396237p3401023.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:52am UTC](https://discuss.elastic.co/t/network-latency-while-adding-new-replica/5521/7 "2017-07-06T03:52:35Z")
Closed on July 6, 2017, 3:52am UTC