# Multimaster? **URL:** \ **Category:** Elasticsearch\ **Created:** [December 30, 2011, 2:24pm UTC](https://discuss.elastic.co/t/multimaster/6259 "2011-12-30T14:24:12Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![project2501](https://avatars.discourse-cdn.com/v4/letter/p/41988e/32.png) [@project2501](https://discuss.elastic.co/u/project2501)\ **Post date:** [December 30, 2011, 2:24pm UTC](https://discuss.elastic.co/t/multimaster/6259/1 "2011-12-30T14:24:12Z")
Hi, Any plans for multimaster capability allowing high volume writes to local nodes yet query across them as one? thanks. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 3:07pm UTC](https://discuss.elastic.co/t/multimaster/6259/2 "2011-12-30T15:07:57Z")
Not sure I understand the question... . When you create an index, you specify the number of shards, and number of replicas. By default its 5 shards and 1 replica for each shard. So effectively each index is "multi mastered". Note, I suspect that you are asking about multi master of an "index". On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5150@gmail.com](mailto:darreng5150@gmail.com) wrote: > Hi, > Any plans for multimaster capability allowing high volume writes to > local nodes yet query across them as one? > > thanks. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 30, 2011, 3:25pm UTC](https://discuss.elastic.co/t/multimaster/6259/3 "2011-12-30T15:25:25Z")
I think that he is talking about primary shards (where Lucene does the indexing job) vs. secondary shards (where you can perform searches). There could be only one shard elected as primary in an Elasticsearch Cluster. So Indexing with Lucene is done in only one place for a given shard instead. Shay, if I understood your berlin's talk, that's the way ES works for indexing. Is that what you were talking about ? David. Le 30 décembre 2011 à 16:07, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a écrit : > Not sure I understand the question... . When you create an index, you > specify the number of shards, and number of replicas. By default its 5 > shards and 1 replica for each shard. So effectively each index is "multi > mastered". > > Note, I suspect that you are asking about multi master of an "index". > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5150@gmail.com](mailto:darreng5150@gmail.com) wrote: > > > Hi, > > Any plans for multimaster capability allowing high volume writes to > > local nodes yet query across them as one? > > > > thanks. > > -- > > David Pilato > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > Twitter : @dadoonet ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 3:31pm UTC](https://discuss.elastic.co/t/multimaster/6259/4 "2011-12-30T15:31:21Z")
Thats not entirely true on how elasticsearch works. Basically, an index is broken down into "replica sets" or partitions. By default, that number if 5. Each partition has at least one shard, and possible extra replicas. A partition has an elected "primary" shard that all indexed go through it, but then also applied to any replica shards. Here is a simple video showing how it works: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram.html). And the berlin buzzwords talk that goes into more depth: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. . On Fri, Dec 30, 2011 at 5:25 PM, [david@pilato.fr](mailto:david@pilato.fr) [david@pilato.fr](mailto:david@pilato.fr) wrote: > \*\* > > I think that he is talking about primary shards (where Lucene does the > indexing job) vs. secondary shards (where you can perform searches). > > There could be only one shard elected as primary in an Elasticsearch > Cluster. > > So Indexing with Lucene is done in only one place for a given shard > instead. > > Shay, if I understood your berlin's talk, that's the way ES works for > indexing. > > Is that what you were talking about ? > > David. > > Le 30 décembre 2011 à 16:07, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) a écrit : > > > Not sure I understand the question... . When you create an index, you > > specify the number of shards, and number of replicas. By default its 5 > > shards and 1 replica for each shard. So effectively each index is "multi > > mastered". > > > > Note, I suspect that you are asking about multi master of an "index". > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5150@gmail.com](mailto:darreng5150@gmail.com) > > wrote: > > > > > Hi, > > > Any plans for multimaster capability allowing high volume writes to > > > local nodes yet query across them as one? > > > > > > thanks. > > -- > David Pilato > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > Twitter : @dadoonet ---
**Author:** ![project2501](https://avatars.discourse-cdn.com/v4/letter/p/41988e/32.png) [@project2501](https://discuss.elastic.co/u/project2501)\ **Post date:** [December 30, 2011, 3:45pm UTC](https://discuss.elastic.co/t/multimaster/6259/5 "2011-12-30T15:45:57Z")
That part I get. But when write documents to the index, I write them to a single known node? Then the indexed data is distributed? Is this true? So would this be a single master in terms of writing/indexing? In my use case, I have 100 servers each with 50 threads all needing to perform writes. That's 5000 index writes per second. When I say multimaster, I mean the ability to perform those index writes to local nodes thereby only 50 threads per node performing writes. Yet, gain the ability to do distributed searches as ES normally would. Does this make more sense? On Dec 30, 10:31 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > Thats not entirely true on how elasticsearch works. Basically, an index is > broken down into "replica sets" or partitions. By default, that number if > 5. Each partition has at least one shard, and possible extra replicas. A > partition has an elected "primary" shard that all indexed go through it, > but then also applied to any replica shards. > > Here is a simple video showing how it works:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram).... > > And the berlin buzzwords talk that goes into more depth:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-)... > . > > On Fri, Dec 30, 2011 at 5:25 PM, [da...@pilato.fr](mailto:da...@pilato.fr) [da...@pilato.fr](mailto:da...@pilato.fr) wrote: > > > \*\* > > > I think that he is talking about primary shards (where Lucene does the > > indexing job) vs. secondary shards (where you can perform searches). > > > There could be only one shard elected as primary in an Elasticsearch > > Cluster. > > > So Indexing with Lucene is done in only one place for a given shard > > instead. > > > Shay, if I understood your berlin's talk, that's the way ES works for > > indexing. > > > Is that what you were talking about ? > > > David. > > > Le 30 décembre 2011 à 16:07, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) a écrit : > > > > Not sure I understand the question... . When you create an index, you > > > specify the number of shards, and number of replicas. By default its 5 > > > shards and 1 replica for each shard. So effectively each index is "multi > > > mastered". > > > > Note, I suspect that you are asking about multi master of an "index". > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) > > > wrote: > > > > > Hi, > > > > Any plans for multimaster capability allowing high volume writes to > > > > local nodes yet query across them as one? > > > > > thanks. > > > -- > > David Pilato > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > Twitter : @dadoonet ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 4:18pm UTC](https://discuss.elastic.co/t/multimaster/6259/6 "2011-12-30T16:18:49Z")
To be honest, I don't really understand what you are trying to say... . In elasticsearch writing to a specific node is not really relevant, since it will be redirected to where the shard that needs to be indexed exists on. See the vids, I don't think you "get" that part... On Fri, Dec 30, 2011 at 5:45 PM, project2501 [darreng5150@gmail.com](mailto:darreng5150@gmail.com) wrote: > That part I get. But when write documents to the index, I write them > to a single known node? > Then the indexed data is distributed? Is this true? > > So would this be a single master in terms of writing/indexing? > > In my use case, I have 100 servers each with 50 threads all needing to > perform writes. > That's 5000 index writes per second. When I say multimaster, I mean > the ability to perform > those index writes to local nodes thereby only 50 threads per node > performing writes. > Yet, gain the ability to do distributed searches as ES normally would. > > Does this make more sense? > > On Dec 30, 10:31 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > Thats not entirely true on how elasticsearch works. Basically, an index > > is > > broken down into "replica sets" or partitions. By default, that number if > > 5. Each partition has at least one shard, and possible extra replicas. A > > partition has an elected "primary" shard that all indexed go through it, > > but then also applied to any replica shards. > > > > Here is a simple video showing how it works: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram).... > > > > And the berlin buzzwords talk that goes into more depth: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-)... > > . > > > > On Fri, Dec 30, 2011 at 5:25 PM, [da...@pilato.fr](mailto:da...@pilato.fr) [da...@pilato.fr](mailto:da...@pilato.fr) > > wrote: > > > > > \*\* > > > > > I think that he is talking about primary shards (where Lucene does the > > > indexing job) vs. secondary shards (where you can perform searches). > > > > > There could be only one shard elected as primary in an Elasticsearch > > > Cluster. > > > > > So Indexing with Lucene is done in only one place for a given shard > > > instead. > > > > > Shay, if I understood your berlin's talk, that's the way ES works for > > > indexing. > > > > > Is that what you were talking about ? > > > > > David. > > > > > Le 30 décembre 2011 à 16:07, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) a écrit : > > > > > > Not sure I understand the question... . When you create an index, you > > > > specify the number of shards, and number of replicas. By default its > > > > 5 > > > > shards and 1 replica for each shard. So effectively each index is > > > > "multi > > > > mastered". > > > > > > Note, I suspect that you are asking about multi master of an "index". > > > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) > > > > wrote: > > > > > > > Hi, > > > > > Any plans for multimaster capability allowing high volume writes > > > > > to > > > > > local nodes yet query across them as one? > > > > > > > thanks. > > > > > -- > > > David Pilato > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > Twitter : @dadoonet ---
**Author:** ![Stanislas\_Polu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stanislas_polu/32/2873_2.png) [@Stanislas\_Polu](https://discuss.elastic.co/u/Stanislas_Polu)\ **Post date:** [December 30, 2011, 4:32pm UTC](https://discuss.elastic.co/t/multimaster/6259/7 "2011-12-30T16:32:39Z")
Looks like being able to force the location of each shard and specify the sharding strategy would meet his requirement. Right? -stan On Dec 30, 2011, at 5:18 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > To be honest, I don't really understand what you are trying to say... . In elasticsearch writing to a specific node is not really relevant, since it will be redirected to where the shard that needs to be indexed exists on. See the vids, I don't think you "get" that part... > > On Fri, Dec 30, 2011 at 5:45 PM, project2501 [darreng5150@gmail.com](mailto:darreng5150@gmail.com) wrote: > That part I get. But when write documents to the index, I write them > to a single known node? > Then the indexed data is distributed? Is this true? > > So would this be a single master in terms of writing/indexing? > > In my use case, I have 100 servers each with 50 threads all needing to > perform writes. > That's 5000 index writes per second. When I say multimaster, I mean > the ability to perform > those index writes to local nodes thereby only 50 threads per node > performing writes. > Yet, gain the ability to do distributed searches as ES normally would. > > Does this make more sense? > > On Dec 30, 10:31 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > Thats not entirely true on how elasticsearch works. Basically, an index is > > broken down into "replica sets" or partitions. By default, that number if > > 5. Each partition has at least one shard, and possible extra replicas. A > > partition has an elected "primary" shard that all indexed go through it, > > but then also applied to any replica shards. > > > > Here is a simple video showing how it works:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram).... > > > > And the berlin buzzwords talk that goes into more depth:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-)... > > . > > > > On Fri, Dec 30, 2011 at 5:25 PM, [da...@pilato.fr](mailto:da...@pilato.fr) [da...@pilato.fr](mailto:da...@pilato.fr) wrote: > > > > > \*\* > > > > > I think that he is talking about primary shards (where Lucene does the > > > indexing job) vs. secondary shards (where you can perform searches). > > > > > There could be only one shard elected as primary in an Elasticsearch > > > Cluster. > > > > > So Indexing with Lucene is done in only one place for a given shard > > > instead. > > > > > Shay, if I understood your berlin's talk, that's the way ES works for > > > indexing. > > > > > Is that what you were talking about ? > > > > > David. > > > > > Le 30 décembre 2011 à 16:07, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) a écrit : > > > > > > Not sure I understand the question... . When you create an index, you > > > > specify the number of shards, and number of replicas. By default its 5 > > > > shards and 1 replica for each shard. So effectively each index is "multi > > > > mastered". > > > > > > Note, I suspect that you are asking about multi master of an "index". > > > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) > > > > wrote: > > > > > > > Hi, > > > > > Any plans for multimaster capability allowing high volume writes to > > > > > local nodes yet query across them as one? > > > > > > > thanks. > > > > > -- > > > David Pilato > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > Twitter : @dadoonet ---
**Author:** ![project2501](https://avatars.discourse-cdn.com/v4/letter/p/41988e/32.png) [@project2501](https://discuss.elastic.co/u/project2501)\ **Post date:** [December 30, 2011, 4:36pm UTC](https://discuss.elastic.co/t/multimaster/6259/8 "2011-12-30T16:36:21Z")
What I'm saying is that 5000+ simultaneous connections to a single node's port prior to redirection won't scale for indexing. Will it? I understand the redirection. But there are practical limits to the number of open sockets a "single" port can handle at one time. So a single write "redirector" can run into scaling issues when you have 1000's of connections at once. I believe ES relies on a single port on a single node for index writes (which it thus redirects). Is that true? Multimaster solves this problem. > **[Multi-master replication](https://en.wikipedia.org/wiki/Multi-master_replication)** > > Multi-master replication is a method of database replication which allows data to be stored by a group of computers, and updated by any member of the group. All members are responsive to client data queries. The multi-master replication system is responsible for propagating the data modifications made by each member to the rest of the group and resolving any conflicts that might arise between concurrent changes made by different members. > Multi-master replication can be contrasted with primary-re... In my use case there will be more throttling at index time, than query time. My understanding also is that queries can occur anywhere in the cluster and be consistent. So the problem doesn't lie there. I'm still learning ES though. I read the materials. 🙂 On Dec 30, 11:18 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > To be honest, I don't really understand what you are trying to say... . In > elasticsearch writing to a specific node is not really relevant, since it > will be redirected to where the shard that needs to be indexed exists on. > See the vids, I don't think you "get" that part... > > On Fri, Dec 30, 2011 at 5:45 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) wrote: > > > That part I get. But when write documents to the index, I write them > > to a single known node? > > Then the indexed data is distributed? Is this true? > > > So would this be a single master in terms of writing/indexing? > > > In my use case, I have 100 servers each with 50 threads all needing to > > perform writes. > > That's 5000 index writes per second. When I say multimaster, I mean > > the ability to perform > > those index writes to local nodes thereby only 50 threads per node > > performing writes. > > Yet, gain the ability to do distributed searches as ES normally would. > > > Does this make more sense? > > > On Dec 30, 10:31 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > Thats not entirely true on how elasticsearch works. Basically, an index > > > is > > > broken down into "replica sets" or partitions. By default, that number if > > > 5. Each partition has at least one shard, and possible extra replicas. A > > > partition has an elected "primary" shard that all indexed go through it, > > > but then also applied to any replica shards. > > > > Here is a simple video showing how it works: > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram).... > > > > And the berlin buzzwords talk that goes into more depth: > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-)... > > > . > > > > On Fri, Dec 30, 2011 at 5:25 PM, [da...@pilato.fr](mailto:da...@pilato.fr) [da...@pilato.fr](mailto:da...@pilato.fr) > > > wrote: > > > > > > > \*\* > > > > > I think that he is talking about primary shards (where Lucene does the > > > > indexing job) vs. secondary shards (where you can perform searches). > > > > > There could be only one shard elected as primary in an Elasticsearch > > > > Cluster. > > > > > So Indexing with Lucene is done in only one place for a given shard > > > > instead. > > > > > Shay, if I understood your berlin's talk, that's the way ES works for > > > > indexing. > > > > > Is that what you were talking about ? > > > > > David. > > > > > Le 30 décembre 2011 à 16:07, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) a écrit : > > > > > > Not sure I understand the question... . When you create an index, you > > > > > specify the number of shards, and number of replicas. By default its > > > > > 5 > > > > > shards and 1 replica for each shard. So effectively each index is > > > > > "multi > > > > > mastered". > > > > > > Note, I suspect that you are asking about multi master of an "index". > > > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) > > > > > wrote: > > > > > > > Hi, > > > > > > Any plans for multimaster capability allowing high volume writes > > > > > > to > > > > > > local nodes yet query across them as one? > > > > > > > thanks. > > > > > -- > > > > David Pilato > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > Twitter : @dadoonet ---
**Author:** ![project2501](https://avatars.discourse-cdn.com/v4/letter/p/41988e/32.png) [@project2501](https://discuss.elastic.co/u/project2501)\ **Post date:** [December 30, 2011, 4:39pm UTC](https://discuss.elastic.co/t/multimaster/6259/9 "2011-12-30T16:39:00Z")
Yes. In other words, let me handle the shard writing decisions at index time in such a way that I can control the topology. This could allow all the servers to work in a "shared nothing" state as much as possible. The shard topology can still be globally known but bypassing a single port/server for redirection will greatly improve write scalability. On Dec 30, 11:32 am, Stanislas Polu [polu.stanis...@gmail.com](mailto:polu.stanis...@gmail.com) wrote: > Looks like being able to force the location of each shard and specify the sharding strategy would meet his requirement. Right? > > -stan > > On Dec 30, 2011, at 5:18 PM, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > To be honest, I don't really understand what you are trying to say... . In elasticsearch writing to a specific node is not really relevant, since it will be redirected to where the shard that needs to be indexed exists on. See the vids, I don't think you "get" that part... > > > On Fri, Dec 30, 2011 at 5:45 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) wrote: > > That part I get. But when write documents to the index, I write them > > to a single known node? > > Then the indexed data is distributed? Is this true? > > > So would this be a single master in terms of writing/indexing? > > > In my use case, I have 100 servers each with 50 threads all needing to > > perform writes. > > That's 5000 index writes per second. When I say multimaster, I mean > > the ability to perform > > those index writes to local nodes thereby only 50 threads per node > > performing writes. > > Yet, gain the ability to do distributed searches as ES normally would. > > > Does this make more sense? > > > On Dec 30, 10:31 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > Thats not entirely true on how elasticsearch works. Basically, an index is > > > broken down into "replica sets" or partitions. By default, that number if > > > 5. Each partition has at least one shard, and possible extra replicas. A > > > partition has an elected "primary" shard that all indexed go through it, > > > but then also applied to any replica shards. > > > > Here is a simple video showing how it works:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram).... > > > > And the berlin buzzwords talk that goes into more depth:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-)... > > > . > > > > On Fri, Dec 30, 2011 at 5:25 PM, [da...@pilato.fr](mailto:da...@pilato.fr) [da...@pilato.fr](mailto:da...@pilato.fr) wrote: > > > > > > > \*\* > > > > > I think that he is talking about primary shards (where Lucene does the > > > > indexing job) vs. secondary shards (where you can perform searches). > > > > > There could be only one shard elected as primary in an Elasticsearch > > > > Cluster. > > > > > So Indexing with Lucene is done in only one place for a given shard > > > > instead. > > > > > Shay, if I understood your berlin's talk, that's the way ES works for > > > > indexing. > > > > > Is that what you were talking about ? > > > > > David. > > > > > Le 30 décembre 2011 à 16:07, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) a écrit : > > > > > > Not sure I understand the question... . When you create an index, you > > > > > specify the number of shards, and number of replicas. By default its 5 > > > > > shards and 1 replica for each shard. So effectively each index is "multi > > > > > mastered". > > > > > > Note, I suspect that you are asking about multi master of an "index". > > > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) > > > > > wrote: > > > > > > > Hi, > > > > > > Any plans for multimaster capability allowing high volume writes to > > > > > > local nodes yet query across them as one? > > > > > > > thanks. > > > > > -- > > > > David Pilato > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > Twitter : @dadoonet ---
**Author:** ![Berkay\_Mollamustafao](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)\ **Post date:** [December 30, 2011, 5:31pm UTC](https://discuss.elastic.co/t/multimaster/6259/10 "2011-12-30T17:31:18Z")
Just to note, you could do this currently by using index aliases. You can create multiple indices and create an alias (that combines multiple indices). Different clients can write to different indices in parallel and you can search across all the indices using the alias. Sharding gives you the option to distribute data automatically but the control is also available via aliases and ES capability to search across indices. Regards, Berkay Mollamustafaoglu mberkay on yahoo, google and skype On Fri, Dec 30, 2011 at 11:39 AM, project2501 [darreng5150@gmail.com](mailto:darreng5150@gmail.com) wrote: > Yes. In other words, let me handle the shard writing decisions at > index time in such a way that I can control the topology. This > could allow all the servers to work in a "shared nothing" state as > much as possible. The shard topology can still be globally known but > bypassing a single port/server for redirection will greatly improve > write scalability. > > On Dec 30, 11:32 am, Stanislas Polu [polu.stanis...@gmail.com](mailto:polu.stanis...@gmail.com) wrote: > > > Looks like being able to force the location of each shard and specify > > the sharding strategy would meet his requirement. Right? > > > > -stan > > > > On Dec 30, 2011, at 5:18 PM, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > To be honest, I don't really understand what you are trying to say... > > > . In elasticsearch writing to a specific node is not really relevant, since > > > it will be redirected to where the shard that needs to be indexed exists > > > on. See the vids, I don't think you "get" that part... > > > > > On Fri, Dec 30, 2011 at 5:45 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) > > > wrote: > > > That part I get. But when write documents to the index, I write them > > > to a single known node? > > > Then the indexed data is distributed? Is this true? > > > > > So would this be a single master in terms of writing/indexing? > > > > > In my use case, I have 100 servers each with 50 threads all needing to > > > perform writes. > > > That's 5000 index writes per second. When I say multimaster, I mean > > > the ability to perform > > > those index writes to local nodes thereby only 50 threads per node > > > performing writes. > > > Yet, gain the ability to do distributed searches as ES normally would. > > > > > Does this make more sense? > > > > > On Dec 30, 10:31 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > Thats not entirely true on how elasticsearch works. Basically, an > > > > index is > > > > broken down into "replica sets" or partitions. By default, that > > > > number if > > > > 5. Each partition has at least one shard, and possible extra > > > > replicas. A > > > > partition has an elected "primary" shard that all indexed go through > > > > it, > > > > but then also applied to any replica shards. > > > > > > Here is a simple video showing how it works: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram).... > > > > > > And the berlin buzzwords talk that goes into more depth: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-)... > > > > . > > > > > > On Fri, Dec 30, 2011 at 5:25 PM, [da...@pilato.fr](mailto:da...@pilato.fr) [da...@pilato.fr](mailto:da...@pilato.fr) > > > > wrote: > > > > > > > > > \*\* > > > > > > > I think that he is talking about primary shards (where Lucene does > > > > > the > > > > > indexing job) vs. secondary shards (where you can perform > > > > > searches). > > > > > > > There could be only one shard elected as primary in an > > > > > Elasticsearch > > > > > Cluster. > > > > > > > So Indexing with Lucene is done in only one place for a given shard > > > > > instead. > > > > > > > Shay, if I understood your berlin's talk, that's the way ES works > > > > > for > > > > > indexing. > > > > > > > Is that what you were talking about ? > > > > > > > David. > > > > > > > Le 30 décembre 2011 à 16:07, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) a > > > > > écrit : > > > > > > > > Not sure I understand the question... . When you create an > > > > > > index, you > > > > > > specify the number of shards, and number of replicas. By default > > > > > > its 5 > > > > > > shards and 1 replica for each shard. So effectively each index > > > > > > is "multi > > > > > > mastered". > > > > > > > > Note, I suspect that you are asking about multi master of an > > > > > > "index". > > > > > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 \< > > > > > > [darreng5...@gmail.com](mailto:darreng5...@gmail.com)\> > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > Any plans for multimaster capability allowing high volume > > > > > > > writes to > > > > > > > local nodes yet query across them as one? > > > > > > > > > thanks. > > > > > > > -- > > > > > David Pilato > > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > > Twitter : @dadoonet ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 6:41pm UTC](https://discuss.elastic.co/t/multimaster/6259/11 "2011-12-30T18:41:21Z")
First, 5000 open connections is not that bad, but why 5000? If you have 100 clients, then you will have 100 connections (you can multiplex on them, if you use the Java client, thats what it does). Also, you can round robin across the server cluster nodes. On Fri, Dec 30, 2011 at 6:36 PM, project2501 [darreng5150@gmail.com](mailto:darreng5150@gmail.com) wrote: > What I'm saying is that 5000+ simultaneous connections to a single > node's port prior to redirection won't scale for indexing. Will it? > > I understand the redirection. But there are practical limits to the > number of open sockets a "single" port can handle at one time. > So a single write "redirector" can run into scaling issues when you > have 1000's of connections at once. I believe ES relies on a single > port on a single node for index writes (which it thus redirects). Is > that true? > > Multimaster solves this problem. > > [Multi-master replication - Wikipedia](http://en.wikipedia.org/wiki/Multi-master_replication) > > In my use case there will be more throttling at index time, than query > time. > My understanding also is that queries can occur anywhere in the > cluster and be consistent. So the problem doesn't lie there. > > I'm still learning ES though. I read the materials. 🙂 > > On Dec 30, 11:18 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > To be honest, I don't really understand what you are trying to say... . > > In > > elasticsearch writing to a specific node is not really relevant, since it > > will be redirected to where the shard that needs to be indexed exists on. > > See the vids, I don't think you "get" that part... > > > > On Fri, Dec 30, 2011 at 5:45 PM, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) > > wrote: > > > > > That part I get. But when write documents to the index, I write them > > > to a single known node? > > > Then the indexed data is distributed? Is this true? > > > > > So would this be a single master in terms of writing/indexing? > > > > > In my use case, I have 100 servers each with 50 threads all needing to > > > perform writes. > > > That's 5000 index writes per second. When I say multimaster, I mean > > > the ability to perform > > > those index writes to local nodes thereby only 50 threads per node > > > performing writes. > > > Yet, gain the ability to do distributed searches as ES normally would. > > > > > Does this make more sense? > > > > > On Dec 30, 10:31 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > Thats not entirely true on how elasticsearch works. Basically, an > > > > index > > > > is > > > > broken down into "replica sets" or partitions. By default, that > > > > number if > > > > 5. Each partition has at least one shard, and possible extra > > > > replicas. A > > > > partition has an elected "primary" shard that all indexed go through > > > > it, > > > > but then also applied to any replica shards. > > > > > > Here is a simple video showing how it works: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2010/02/08/es-distributed-diagram).. > > > > .. > > > > > > And the berlin buzzwords talk that goes into more depth: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-). > > > > .. > > > > . > > > > > > On Fri, Dec 30, 2011 at 5:25 PM, [da...@pilato.fr](mailto:da...@pilato.fr) [da...@pilato.fr](mailto:da...@pilato.fr) > > > > wrote: > > > > > > > > > \*\* > > > > > > > I think that he is talking about primary shards (where Lucene does > > > > > the > > > > > indexing job) vs. secondary shards (where you can perform > > > > > searches). > > > > > > > There could be only one shard elected as primary in an > > > > > Elasticsearch > > > > > Cluster. > > > > > > > So Indexing with Lucene is done in only one place for a given shard > > > > > instead. > > > > > > > Shay, if I understood your berlin's talk, that's the way ES works > > > > > for > > > > > indexing. > > > > > > > Is that what you were talking about ? > > > > > > > David. > > > > > > > Le 30 décembre 2011 à 16:07, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) a > > > > > écrit : > > > > > > > > Not sure I understand the question... . When you create an > > > > > > index, you > > > > > > specify the number of shards, and number of replicas. By default > > > > > > its > > > > > > 5 > > > > > > shards and 1 replica for each shard. So effectively each index is > > > > > > "multi > > > > > > mastered". > > > > > > > > Note, I suspect that you are asking about multi master of an > > > > > > "index". > > > > > > > > On Fri, Dec 30, 2011 at 4:24 PM, project2501 \< > > > > > > [darreng5...@gmail.com](mailto:darreng5...@gmail.com)\> > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > Any plans for multimaster capability allowing high volume > > > > > > > writes > > > > > > > to > > > > > > > local nodes yet query across them as one? > > > > > > > > > thanks. > > > > > > > -- > > > > > David Pilato > > > > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > > > > Twitter : @dadoonet ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [December 31, 2011, 7:21am UTC](https://discuss.elastic.co/t/multimaster/6259/12 "2011-12-31T07:21:00Z")
On Dec 30, 5:36 pm, project2501 [darreng5...@gmail.com](mailto:darreng5...@gmail.com) wrote: > I understand the redirection. But there are practical limits to the > number of open sockets a "single" port can handle at one time. > So a single write "redirector" can run into scaling issues when you > have 1000's of connections at once. I believe ES relies on a single > port on a single node for index writes (which it thus redirects). I don't understand the exact issue you seem to be worried about, but do note that you can write to any node in the cluster (which has `http` enabled). The changes will be propagated to all the replicas, quite similar to the “multi-master” scenario (eg. in something like CouchDB). Maybe check out the relevant configs: [https://github.com/elasticsearch/elasticsearch/blob/master/config/elasticsearch.yml#L35](https://github.com/elasticsearch/elasticsearch/blob/master/config/elasticsearch.yml#L35) Karel ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:44am UTC](https://discuss.elastic.co/t/multimaster/6259/13 "2017-07-06T03:44:05Z")