# Elasticsearch replication protocol between datacenters? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 28, 2012, 9:26am UTC](https://discuss.elastic.co/t/elasticsearch-replication-protocol-between-datacenters/6532 "2012-01-28T09:26:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 28, 2012, 9:26am UTC](https://discuss.elastic.co/t/elasticsearch-replication-protocol-between-datacenters/6532/1 "2012-01-28T09:26:14Z")
Hi, how about Bittorrent, could it be a feasible protocol for future Elasticsearch replication between datacenters? Is the idea good or bad? Pros and cons? Any comments welcome. Just stumbled upon this post where Bittorrent protocol is used to overcome solr index replication deficiencies (but within a datacenter I presume) [http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/](http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/) Jörg ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 28, 2012, 10:07am UTC](https://discuss.elastic.co/t/elasticsearch-replication-protocol-between-datacenters/6532/2 "2012-01-28T10:07:31Z")
That's a very interesting article. They decrease the replication time from 60 mn to 6 mn using bittorrent instead of http protocol ! That's really significant. David Le 28 janv. 2012 à 10:26, jprante [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit : > Hi, > > how about Bittorrent, could it be a feasible protocol for future > Elasticsearch replication between datacenters? Is the idea good or > bad? Pros and cons? Any comments welcome. > > Just stumbled upon this post where Bittorrent protocol is used to > overcome solr index replication deficiencies (but within a datacenter > I presume) > > [http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/](http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/) > > Jörg ---
**Author:** ![Paul\_Loy](https://avatars.discourse-cdn.com/v4/letter/p/ad7895/32.png) [@Paul\_Loy](https://discuss.elastic.co/u/Paul_Loy)\ **Post date:** [January 29, 2012, 9:32am UTC](https://discuss.elastic.co/t/elasticsearch-replication-protocol-between-datacenters/6532/3 "2012-01-29T09:32:29Z")
We use BitTorrent for our deploys. It has reduced deployment time to 10% for us also. But I think this is where Solr vs Elasticsearch becomes interesting. Solr, if I am correct, is Master-slave. That is one index is replicated but not sharded. ES is sharded and replicated so individual replicas of individual shards are much smaller than the entire index - well 1/shards times smaller. Second, I think ES pushes changes to replicas as deltas already rather than needing BitTorrent to hash and tell us what has changed. So in the general usage pattern I don't think it'll help out that much. But what about failure cases? Perhaps when you spin up a new node, that's when a BitTorrent protocol could make you some savings in ES? Paul. On Sat, Jan 28, 2012 at 2:07 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > That's a very interesting article. > They decrease the replication time from 60 mn to 6 mn using bittorrent > instead of http protocol ! > That's really significant. > > David > > Le 28 janv. 2012 à 10:26, jprante [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit : > > > Hi, > > > > how about Bittorrent, could it be a feasible protocol for future > > Elasticsearch replication between datacenters? Is the idea good or > > bad? Pros and cons? Any comments welcome. > > > > Just stumbled upon this post where Bittorrent protocol is used to > > overcome solr index replication deficiencies (but within a datacenter > > I presume) > > [http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/](http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/) > > > Jörg ## -- Paul Loy [paul@keteracel.com](mailto:paul@keteracel.com) [http://uk.linkedin.com/in/paulloy](http://uk.linkedin.com/in/paulloy) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 29, 2012, 6:25pm UTC](https://discuss.elastic.co/t/elasticsearch-replication-protocol-between-datacenters/6532/4 "2012-01-29T18:25:46Z")
Replication in elasticsearch is different than current Solr replication mode, it does not replica internal index segments, where bit torrent can make some sense, it replicates operations. On Sunday, January 29, 2012 at 11:32 AM, Paul Loy wrote: > We use BitTorrent for our deploys. It has reduced deployment time to 10% for us also. > > But I think this is where Solr vs Elasticsearch becomes interesting. Solr, if I am correct, is Master-slave. That is one index is replicated but not sharded. ES is sharded and replicated so individual replicas of individual shards are much smaller than the entire index - well 1/shards times smaller. > > Second, I think ES pushes changes to replicas as deltas already rather than needing BitTorrent to hash and tell us what has changed. > > So in the general usage pattern I don't think it'll help out that much. But what about failure cases? Perhaps when you spin up a new node, that's when a BitTorrent protocol could make you some savings in ES? > > Paul. > > On Sat, Jan 28, 2012 at 2:07 AM, David Pilato \<[david@pilato.fr](mailto:david@pilato.fr) ([mailto:david@pilato.fr](mailto:david@pilato.fr))\> wrote: > > > That's a very interesting article. > > They decrease the replication time from 60 mn to 6 mn using bittorrent instead of http protocol ! > > That's really significant. > > > > David > > > > Le 28 janv. 2012 à 10:26, jprante \<[joergprante@gmail.com](mailto:joergprante@gmail.com) ([mailto:joergprante@gmail.com](mailto:joergprante@gmail.com))\> a écrit : > > > > > Hi, > > > > > > how about Bittorrent, could it be a feasible protocol for future > > > Elasticsearch replication between datacenters? Is the idea good or > > > bad? Pros and cons? Any comments welcome. > > > > > > Just stumbled upon this post where Bittorrent protocol is used to > > > overcome solr index replication deficiencies (but within a datacenter > > > I presume) > > > > > > [http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/](http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/) > > > > > > Jörg > > ## -- > > Paul Loy > [paul@keteracel.com](mailto:paul@keteracel.com) ([mailto:paul@keteracel.com](mailto:paul@keteracel.com)) > [http://uk.linkedin.com/in/paulloy](http://uk.linkedin.com/in/paulloy) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:41am UTC](https://discuss.elastic.co/t/elasticsearch-replication-protocol-between-datacenters/6532/5 "2017-07-06T03:41:07Z")