# Elasticsearch client only cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [February 24, 2013, 8:44pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869 "2013-02-24T20:44:28Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [February 24, 2013, 8:44pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/1 "2013-02-24T20:44:28Z")
I'm using elasticsearch for logstash and I would like to create a client only (30 node)cluster with 0 shards are distributed and no replication. I have tons of logs so have configured logging per rack and I want to be able to connect to any es instance to search all the cluster nodes. This setup has worked but when nodes get restarted it seems to affect the cluster health which causes ":exception=\>org.elasticsearch.action.UnavailableShardsException". My current setting per es instances is unicast discovery shards=5 replication=0 current health { "cluster\_name" : "elasticsearch", "status" : "red", "timed\_out" : false, "number\_of\_nodes" : 52, "number\_of\_data\_nodes" : 26, "active\_primary\_shards" : 22, "active\_shards" : 22, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 3 } If I lose a few nodes I don't want it to affect the rest from starting up or processing correctly. Am I missing any settings for a client only cluster? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 25, 2013, 10:10am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/2 "2013-02-25T10:10:27Z")
On Sun, 2013-02-24 at 12:44 -0800, [beharris@gmail.com](mailto:beharris@gmail.com) wrote: > I'm using elasticsearch for logstash and I would like to create a > client only (30 node)cluster with 0 shards are distributed and no > replication. You need to have shards to index data. Shards are not just for replication. Without any shards, you can't index any data. Do you mean that you don't want any replica shards? You can control the number of primary shards that an index contains when you create the index, and the number\_of\_replicas can be updated at any time. clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [February 25, 2013, 6:09pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/3 "2013-02-25T18:09:12Z")
Sorry I meant I would like to create a 30 node cluster one per rack and ensure that the 1 shard stays within the rack. Is that possible within elasticsearch? The cluster would be only used o search one instance and look at all data without having data get replicated or sharded between racks. On Monday, February 25, 2013 2:10:27 AM UTC-8, Clinton Gormley wrote: > On Sun, 2013-02-24 at 12:44 -0800, [beha...@gmail.com](mailto:beha...@gmail.com) \ wrote: > > > I'm using elasticsearch for logstash and I would like to create a > > client only (30 node)cluster with 0 shards are distributed and no > > replication. > > You need to have shards to index data. Shards are not just for > replication. Without any shards, you can't index any data. > > Do you mean that you don't want any replica shards? You can control the > number of primary shards that an index contains when you create the > index, and the number\_of\_replicas can be updated at any time. > > clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 25, 2013, 8:31pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/4 "2013-02-25T20:31:01Z")
On Mon, 2013-02-25 at 10:09 -0800, [beharris@gmail.com](mailto:beharris@gmail.com) wrote: > Sorry I meant I would like to create a 30 node cluster one per rack > and ensure that the 1 shard stays within the rack. > Is that possible within elasticsearch? > > The cluster would be only used o search one instance and look at all > data without having data get replicated or sharded between racks. Sorry but I still have no idea what you're trying to achieve. One node per rack? You have 30 racks? /me is lost Perhaps some more detail... clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [February 26, 2013, 12:23am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/5 "2013-02-26T00:23:55Z")
So I have 30 racks at a colo and have 1 es instance per rack. The 1 es instance per rack is used to index all logs for that rack only. Is there anyway for each instance to join a 30 node cluster as a (client only) and not replicate or shard data between them? The purpose of the client only cluster would allow me to search one instance and have es query all members for data. On Monday, February 25, 2013 12:31:01 PM UTC-8, Clinton Gormley wrote: > On Mon, 2013-02-25 at 10:09 -0800, [beha...@gmail.com](mailto:beha...@gmail.com) \ wrote: > > > Sorry I meant I would like to create a 30 node cluster one per rack > > and ensure that the 1 shard stays within the rack. > > Is that possible within elasticsearch? > > > > The cluster would be only used o search one instance and look at all > > data without having data get replicated or sharded between racks. > > Sorry but I still have no idea what you're trying to achieve. One node > per rack? You have 30 racks? /me is lost > > Perhaps some more detail... > > clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 26, 2013, 9:41am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/6 "2013-02-26T09:41:46Z")
Hiya OK, the picture is slowly evolving 🙂 On Mon, 2013-02-25 at 16:23 -0800, [beharris@gmail.com](mailto:beharris@gmail.com) wrote: > So I have 30 racks at a colo and have 1 es instance per rack. > The 1 es instance per rack is used to index all logs for that rack > only. > > Is there anyway for each instance to join a 30 node cluster as a > (client only) and not replicate or shard data between them? A "client" in Elasticsearch terminology doesn't hold any data. Hence part of the confusion. I think what you're asking is: Can I have an index on a single node in the cluster? The answer is yes: you can create 30 indices, and specify rack awareness for each index, so that each index sits in a single rack. > The purpose of the client only cluster would allow me to search one > instance and have es query all members for data. Yes, you can connect to any node in the cluster and query one or more indices. It will forward queries to all relevant nodes. Note: I don't recommend this setup. Especially with 30 nodes, the chances of one of them going down is pretty high. Hardware fails. With your current setup (esp if you don't have any replicas) then you run a good chance of losing data. Why not just use Elasticsearch as the distributed system that it is intended to be? clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [February 26, 2013, 6:12pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/7 "2013-02-26T18:12:28Z")
I'm using elasticsearch for logstash and we generate tons of logs per rack. I would like to keep the data local to the rack so the traffic doesn't need to cross switches. I don't have an immediate concern with failure or losing any data so I'm ok with losing a node and can just reprocess the logs for now. If I use the distributed system, can I force data to stay local to the rack? On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton Gormley wrote: > Hiya > > OK, the picture is slowly evolving 🙂 > > On Mon, 2013-02-25 at 16:23 -0800, [beha...@gmail.com](mailto:beha...@gmail.com) \ wrote: > > > So I have 30 racks at a colo and have 1 es instance per rack. > > The 1 es instance per rack is used to index all logs for that rack > > only. > > > > Is there anyway for each instance to join a 30 node cluster as a > > (client only) and not replicate or shard data between them? > > A "client" in Elasticsearch terminology doesn't hold any data. Hence > part of the confusion. I think what you're asking is: Can I have an > index on a single node in the cluster? > > The answer is yes: you can create 30 indices, and specify rack awareness > for each index, so that each index sits in a single rack. > > > The purpose of the client only cluster would allow me to search one > > instance and have es query all members for data. > > Yes, you can connect to any node in the cluster and query one or more > indices. It will forward queries to all relevant nodes. > > Note: I don't recommend this setup. Especially with 30 nodes, the > chances of one of them going down is pretty high. Hardware fails. With > your current setup (esp if you don't have any replicas) then you run a > good chance of losing data. > > Why not just use Elasticsearch as the distributed system that it is > intended to be? > > clint -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 26, 2013, 6:40pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/8 "2013-02-26T18:40:36Z")
> If I use the distributed system, can I force data to stay local to the > rack? Yes. Look for rack awareness in the docs > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton Gormley wrote: > Hiya > > ``` > OK, the picture is slowly evolving :) > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com wrote: > > So I have 30 racks at a colo and have 1 es instance per > rack. > > The 1 es instance per rack is used to index all logs for > that rack > > only. > > > > Is there anyway for each instance to join a 30 node cluster > as a > > (client only) and not replicate or shard data between them? > > A "client" in Elasticsearch terminology doesn't hold any > data. Hence > part of the confusion. I think what you're asking is: Can I > have an > index on a single node in the cluster? > > The answer is yes: you can create 30 indices, and specify rack > awareness > for each index, so that each index sits in a single rack. > > > > The purpose of the client only cluster would allow me to > search one > > instance and have es query all members for data. > > Yes, you can connect to any node in the cluster and query one > or more > indices. It will forward queries to all relevant nodes. > > Note: I don't recommend this setup. Especially with 30 nodes, > the > chances of one of them going down is pretty high. Hardware > fails. With > your current setup (esp if you don't have any replicas) then > you run a > good chance of losing data. > > Why not just use Elasticsearch as the distributed system that > it is > intended to be? > > clint > > ``` > > -- > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [February 27, 2013, 8:16am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/9 "2013-02-27T08:16:06Z")
Great, I checked out the docs and can came up with following config to enable rack awareness for each instance that would look something like this. node: name: node1 rack\_id: rack1 cluster: name: elasticsearch routing: allocation: awareness: attributes: rack\_id Since logstash creates an index per day(logstash-2013.02.26) per node, would I have to do anything special to make sure that an index and it's shards are created for each node individually? Currently only 5 shards are allocated in my shard=5 rep=0 cluster receiving logs for +1k servers under low load. The goal is to have all 30 nodes to have an index started and receiving logs for its rack to handle high load. > es shards -v > index shard pri/rep state docs size > bytes node > logstash-2013.02.26 0 p STARTED 28031927 22.9gb 24677604861 > n7 > logstash-2013.02.26 1 p STARTED 26853297 22gb 23641399741 > n18 > logstash-2013.02.26 2 p STARTED 28035826 22.9gb 24686606451 > n21 > logstash-2013.02.26 3 p STARTED 28033599 22.9gb 24695469792 > n24 > logstash-2013.02.26 4 p STARTED 28037600 22.9gb 24687686161 > n5 Any recommendations on how to configure elasticsearch rack awareness and routing to handle this. On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley wrote: > > If I use the distributed system, can I force data to stay local to the > > rack? > > Yes. Look for rack awareness in the docs > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton Gormley wrote: > > Hiya > > > > ``` > > OK, the picture is slowly evolving :) > > > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com wrote: > > > So I have 30 racks at a colo and have 1 es instance per > > rack. > > > The 1 es instance per rack is used to index all logs for > > that rack > > > only. > > > > > > Is there anyway for each instance to join a 30 node cluster > > as a > > > (client only) and not replicate or shard data between them? > > > > A "client" in Elasticsearch terminology doesn't hold any > > data. Hence > > part of the confusion. I think what you're asking is: Can I > > have an > > index on a single node in the cluster? > > > > The answer is yes: you can create 30 indices, and specify rack > > awareness > > for each index, so that each index sits in a single rack. > > > > > > The purpose of the client only cluster would allow me to > > search one > > > instance and have es query all members for data. > > > > Yes, you can connect to any node in the cluster and query one > > or more > > indices. It will forward queries to all relevant nodes. > > > > Note: I don't recommend this setup. Especially with 30 nodes, > > the > > chances of one of them going down is pretty high. Hardware > > fails. With > > your current setup (esp if you don't have any replicas) then > > you run a > > good chance of losing data. > > > > Why not just use Elasticsearch as the distributed system that > > it is > > intended to be? > > > > clint > > > > ``` > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 27, 2013, 10:58am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/10 "2013-02-27T10:58:10Z")
On Wed, 2013-02-27 at 00:16 -0800, B wrote: > Great, I checked out the docs and can came up with following config to > enable rack awareness for each instance that would look something like > this. > > node: > name: node1 > rack\_id: rack1 > cluster: > name: elasticsearch > routing: > allocation: > awareness: > attributes: rack\_id > > Since logstash creates an index per day(logstash-2013.02.26) per node, > would I have to do anything special to make sure that an index and > it's shards are created for each node individually? If you want 30 indices on 30 different nodes, then you need to create each index with a different name, and set the allocation on each index to tie it to a single node. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. For instance, you can use index templates to say: if the index name matches "node\_1\_\*" then set index.routing.allocation.include.rack\_id to "node\_1" > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. clint > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > receiving logs for +1k servers under low load. The goal is to have all > 30 nodes to have an index started and receiving logs for its rack to > handle high load. > > > es shards -v > > index shard pri/rep state docs size > > bytes node > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > 24677604861 n7 > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > 23641399741 n18 > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > 24686606451 n21 > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > 24695469792 n24 > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > 24687686161 n5 > > Any recommendations on how to configure elasticsearch rack awareness > and routing to handle this. > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > wrote: > > ``` > > If I use the distributed system, can I force data to stay > local to the > > rack? > > Yes. Look for rack awareness in the docs > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > Gormley wrote: > > Hiya > > > > OK, the picture is slowly evolving :) > > > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com > wrote: > > > So I have 30 racks at a colo and have 1 es > instance per > > rack. > > > The 1 es instance per rack is used to index all > logs for > > that rack > > > only. > > > > > > Is there anyway for each instance to join a 30 > node cluster > > as a > > > (client only) and not replicate or shard data > between them? > > > > A "client" in Elasticsearch terminology doesn't hold > any > > data. Hence > > part of the confusion. I think what you're asking > is: Can I > > have an > > index on a single node in the cluster? > > > > The answer is yes: you can create 30 indices, and > specify rack > > awareness > > for each index, so that each index sits in a single > rack. > > > > > > The purpose of the client only cluster would allow > me to > > search one > > > instance and have es query all members for data. > > > > Yes, you can connect to any node in the cluster and > query one > > or more > > indices. It will forward queries to all relevant > nodes. > > > > Note: I don't recommend this setup. Especially with > 30 nodes, > > the > > chances of one of them going down is pretty high. > Hardware > > fails. With > > your current setup (esp if you don't have any > replicas) then > > you run a > > good chance of losing data. > > > > Why not just use Elasticsearch as the distributed > system that > > it is > > intended to be? > > > > clint > > > > > > > > -- > > You received this message because you are subscribed to the > Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails > from it, send > > an email to elasticsearc...@googlegroups.com. > > For more options, visit > https://groups.google.com/groups/opt_out. > > > > > > ``` > > -- > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [February 28, 2013, 7:43am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/11 "2013-02-28T07:43:05Z")
ok, just playing around with 28 nodes I set shards=28 and replicas=0. I now have 28 shards all of my nodes now which is balanced. This is definitely an alternate setup that will work but I will still to try to find out how to keep logs local to the rack using the templates you listed below. Perhaps I don't need to keep logs per rack as this seems to balance out the storage pretty well. What mechanism does elasticsearch use to keep the data balanced across all nodes? 28 shards all reported on all nodes. logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 .. logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > Great, I checked out the docs and can came up with following config to > > enable rack awareness for each instance that would look something like > > this. > > > > node: > > name: node1 > > rack\_id: rack1 > > cluster: > > name: elasticsearch > > routing: > > allocation: > > awareness: > > attributes: rack\_id > > > > Since logstash creates an index per day(logstash-2013.02.26) per node, > > would I have to do anything special to make sure that an index and > > it's shards are created for each node individually? > > If you want 30 indices on 30 different nodes, then you need to create > each index with a different name, and set the allocation on each index > to tie it to a single node. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > For instance, you can use index templates to say: if the index name > matches "node\_1\_\*" then set index.routing.allocation.include.rack\_id to > "node\_1" > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > clint > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > receiving logs for +1k servers under low load. The goal is to have all > > 30 nodes to have an index started and receiving logs for its rack to > > handle high load. > > > > > es shards -v > > > index shard pri/rep state docs size > > > bytes node > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > 24677604861 n7 > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > 23641399741 n18 > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > 24686606451 n21 > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > 24695469792 n24 > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > 24687686161 n5 > > > > Any recommendations on how to configure elasticsearch rack awareness > > and routing to handle this. > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > wrote: > > > > ``` > > > If I use the distributed system, can I force data to stay > > local to the > > > rack? > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > Gormley wrote: > > > Hiya > > > > > > OK, the picture is slowly evolving :) > > > > > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com > > wrote: > > > > So I have 30 racks at a colo and have 1 es > > instance per > > > rack. > > > > The 1 es instance per rack is used to index all > > logs for > > > that rack > > > > only. > > > > > > > > Is there anyway for each instance to join a 30 > > node cluster > > > as a > > > > (client only) and not replicate or shard data > > between them? > > > > > > A "client" in Elasticsearch terminology doesn't hold > > any > > > data. Hence > > > part of the confusion. I think what you're asking > > is: Can I > > > have an > > > index on a single node in the cluster? > > > > > > The answer is yes: you can create 30 indices, and > > specify rack > > > awareness > > > for each index, so that each index sits in a single > > rack. > > > > > > > > The purpose of the client only cluster would allow > > me to > > > search one > > > > instance and have es query all members for data. > > > > > > Yes, you can connect to any node in the cluster and > > query one > > > or more > > > indices. It will forward queries to all relevant > > nodes. > > > > > > Note: I don't recommend this setup. Especially with > > 30 nodes, > > > the > > > chances of one of them going down is pretty high. > > Hardware > > > fails. With > > > your current setup (esp if you don't have any > > replicas) then > > > you run a > > > good chance of losing data. > > > > > > Why not just use Elasticsearch as the distributed > > system that > > > it is > > > intended to be? > > > > > > clint > > > > > > > > > > > > -- > > > You received this message because you are subscribed to the > > Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails > > from it, send > > > an email to elasticsearc...@googlegroups.com. > > > For more options, visit > > https://groups.google.com/groups/opt_out. > > > > > > > > > > ``` > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [March 6, 2013, 6:56am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/12 "2013-03-06T06:56:53Z")
Clinton, if i keep to the distributed model of a 28 node cluster with all using the same index. Would shards=14 , repl=1 be a wise choice to keep all nodes doing something? Right now testing shards=28 and repl=0 which is working great and the cluster is balanced with data and load but thinking long term of node failures and adding nodes in the future. On Feb 27, 2013 11:43 PM, "Brian Harris" [beharris@gmail.com](mailto:beharris@gmail.com) wrote: > ok, just playing around with 28 nodes I set shards=28 and replicas=0. > > I now have 28 shards all of my nodes now which is balanced. > This is definitely an alternate setup that will work but I will still to > try to find out how to keep logs local to the rack using the templates you > listed below. > > Perhaps I don't need to keep logs per rack as this seems to balance out > the storage pretty well. > What mechanism does elasticsearch use to keep the data balanced across all > nodes? > > 28 shards all reported on all nodes. > logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 > .. > logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 > > On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com)wrote: > > > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > > > Great, I checked out the docs and can came up with following config to > > > enable rack awareness for each instance that would look something like > > > this. > > > > > > node: > > > name: node1 > > > rack\_id: rack1 > > > cluster: > > > name: elasticsearch > > > routing: > > > allocation: > > > awareness: > > > attributes: rack\_id > > > > > > Since logstash creates an index per day(logstash-2013.02.26) per node, > > > would I have to do anything special to make sure that an index and > > > it's shards are created for each node individually? > > > > If you want 30 indices on 30 different nodes, then you need to create > > each index with a different name, and set the allocation on each index > > to tie it to a single node. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > For instance, you can use index templates to say: if the index name > > matches "node\_1\_\*" then set index.routing.allocation.include.rack\_id to > > "node\_1" > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > clint > > > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > > receiving logs for +1k servers under low load. The goal is to have all > > > 30 nodes to have an index started and receiving logs for its rack to > > > handle high load. > > > > > > > es shards -v > > > > index shard pri/rep state docs size > > > > bytes node > > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > > 24677604861 n7 > > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > > 23641399741 n18 > > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > > 24686606451 n21 > > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > > 24695469792 n24 > > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > > 24687686161 n5 > > > > > > Any recommendations on how to configure elasticsearch rack awareness > > > and routing to handle this. > > > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > > wrote: > > > > > > ``` > > > > If I use the distributed system, can I force data to stay > > > local to the > > > > rack? > > > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > > Gormley wrote: > > > > Hiya > > > > > > > > OK, the picture is slowly evolving :) > > > > > > > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com > > > wrote: > > > > > So I have 30 racks at a colo and have 1 es > > > instance per > > > > rack. > > > > > The 1 es instance per rack is used to index all > > > logs for > > > > that rack > > > > > only. > > > > > > > > > > Is there anyway for each instance to join a 30 > > > node cluster > > > > as a > > > > > (client only) and not replicate or shard data > > > between them? > > > > > > > > A "client" in Elasticsearch terminology doesn't hold > > > any > > > > data. Hence > > > > part of the confusion. I think what you're asking > > > is: Can I > > > > have an > > > > index on a single node in the cluster? > > > > > > > > The answer is yes: you can create 30 indices, and > > > specify rack > > > > awareness > > > > for each index, so that each index sits in a single > > > rack. > > > > > > > > > > The purpose of the client only cluster would allow > > > me to > > > > search one > > > > > instance and have es query all members for data. > > > > > > > > Yes, you can connect to any node in the cluster and > > > query one > > > > or more > > > > indices. It will forward queries to all relevant > > > nodes. > > > > > > > > Note: I don't recommend this setup. Especially with > > > 30 nodes, > > > > the > > > > chances of one of them going down is pretty high. > > > Hardware > > > > fails. With > > > > your current setup (esp if you don't have any > > > replicas) then > > > > you run a > > > > good chance of losing data. > > > > > > > > Why not just use Elasticsearch as the distributed > > > system that > > > > it is > > > > intended to be? > > > > > > > > clint > > > > > > > > > > > > > > > > -- > > > > You received this message because you are subscribed to the > > > Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails > > > from it, send > > > > an email to elasticsearc...@googlegroups.com. > > > > For more options, visit > > > https://groups.google.com/groups/opt_out. > > > > > > > > > > > > > > ``` > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [March 6, 2013, 11:39am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/13 "2013-03-06T11:39:52Z")
Hi, 0 replicas is risky. 1 node dies and you are missing ~1/28th of your data. So repl \> 0 is definitely better. ## Otis ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) On Wednesday, March 6, 2013 1:56:53 AM UTC-5, B wrote: > Clinton, if i keep to the distributed model of a 28 node cluster with all > using the same index. Would shards=14 , repl=1 be a wise choice to keep all > nodes doing something? > > Right now testing shards=28 and repl=0 which is working great and the > cluster is balanced with data and load but thinking long term of node > failures and adding nodes in the future. > On Feb 27, 2013 11:43 PM, "Brian Harris" \<[beha...@gmail.com](mailto:beha...@gmail.com) \\> > wrote: > > > ok, just playing around with 28 nodes I set shards=28 and replicas=0. > > > > I now have 28 shards all of my nodes now which is balanced. > > This is definitely an alternate setup that will work but I will still to > > try to find out how to keep logs local to the rack using the templates you > > listed below. > > > > Perhaps I don't need to keep logs per rack as this seems to balance out > > the storage pretty well. > > What mechanism does elasticsearch use to keep the data balanced across > > all nodes? > > > > 28 shards all reported on all nodes. > > logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 > > .. > > logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 > > > > On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley \<[cl...@traveljury.com](mailto:cl...@traveljury.com)\ > > > > > wrote: > > > > > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > > > > > Great, I checked out the docs and can came up with following config to > > > > enable rack awareness for each instance that would look something like > > > > this. > > > > > > > > node: > > > > name: node1 > > > > rack\_id: rack1 > > > > cluster: > > > > name: elasticsearch > > > > routing: > > > > allocation: > > > > awareness: > > > > attributes: rack\_id > > > > > > > > Since logstash creates an index per day(logstash-2013.02.26) per node, > > > > would I have to do anything special to make sure that an index and > > > > it's shards are created for each node individually? > > > > > > If you want 30 indices on 30 different nodes, then you need to create > > > each index with a different name, and set the allocation on each index > > > to tie it to a single node. > > > > > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > > > For instance, you can use index templates to say: if the index name > > > matches "node\_1\_\*" then set index.routing.allocation.include.rack\_id to > > > "node\_1" > > > > > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > > > clint > > > > > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > > > receiving logs for +1k servers under low load. The goal is to have all > > > > 30 nodes to have an index started and receiving logs for its rack to > > > > handle high load. > > > > > > > > > es shards -v > > > > > index shard pri/rep state docs size > > > > > bytes node > > > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > > > 24677604861 n7 > > > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > > > 23641399741 n18 > > > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > > > 24686606451 n21 > > > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > > > 24695469792 n24 > > > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > > > 24687686161 n5 > > > > > > > > Any recommendations on how to configure elasticsearch rack awareness > > > > and routing to handle this. > > > > > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > > > wrote: > > > > > > > > ``` > > > > > If I use the distributed system, can I force data to stay > > > > local to the > > > > > rack? > > > > > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > > > Gormley wrote: > > > > > Hiya > > > > > > > > > > OK, the picture is slowly evolving :) > > > > > > > > > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com > > > > wrote: > > > > > > So I have 30 racks at a colo and have 1 es > > > > instance per > > > > > rack. > > > > > > The 1 es instance per rack is used to index all > > > > logs for > > > > > that rack > > > > > > only. > > > > > > > > > > > > Is there anyway for each instance to join a 30 > > > > node cluster > > > > > as a > > > > > > (client only) and not replicate or shard data > > > > between them? > > > > > > > > > > A "client" in Elasticsearch terminology doesn't hold > > > > any > > > > > data. Hence > > > > > part of the confusion. I think what you're asking > > > > is: Can I > > > > > have an > > > > > index on a single node in the cluster? > > > > > > > > > > The answer is yes: you can create 30 indices, and > > > > specify rack > > > > > awareness > > > > > for each index, so that each index sits in a single > > > > rack. > > > > > > > > > > > > The purpose of the client only cluster would allow > > > > me to > > > > > search one > > > > > > instance and have es query all members for data. > > > > > > > > > > Yes, you can connect to any node in the cluster and > > > > query one > > > > > or more > > > > > indices. It will forward queries to all relevant > > > > nodes. > > > > > > > > > > Note: I don't recommend this setup. Especially with > > > > 30 nodes, > > > > > the > > > > > chances of one of them going down is pretty high. > > > > Hardware > > > > > fails. With > > > > > your current setup (esp if you don't have any > > > > replicas) then > > > > > you run a > > > > > good chance of losing data. > > > > > > > > > > Why not just use Elasticsearch as the distributed > > > > system that > > > > > it is > > > > > intended to be? > > > > > > > > > > clint > > > > > > > > > > > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the > > > > Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails > > > > from it, send > > > > > an email to elasticsearc...@googlegroups.com. > > > > > For more options, visit > > > > https://groups.google.com/groups/opt_out. > > > > > > > > > > > > > > > > > > ``` > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > > . > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [March 7, 2013, 6:58am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/14 "2013-03-07T06:58:06Z")
Otis using 28 nodes with1 daily index what would be the best shard and replication scheme? On Wed, Mar 6, 2013 at 3:39 AM, Otis Gospodnetic \<[otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com) > wrote: > Hi, > > 0 replicas is risky. 1 node dies and you are missing ~1/28th of your > data. So repl \> 0 is definitely better. > > ## Otis > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > On Wednesday, March 6, 2013 1:56:53 AM UTC-5, B wrote: > > > Clinton, if i keep to the distributed model of a 28 node cluster with all > > using the same index. Would shards=14 , repl=1 be a wise choice to keep all > > nodes doing something? > > > > Right now testing shards=28 and repl=0 which is working great and the > > cluster is balanced with data and load but thinking long term of node > > failures and adding nodes in the future. > > On Feb 27, 2013 11:43 PM, "Brian Harris" [beha...@gmail.com](mailto:beha...@gmail.com) wrote: > > > > > ok, just playing around with 28 nodes I set shards=28 and replicas=0. > > > > > > I now have 28 shards all of my nodes now which is balanced. > > > This is definitely an alternate setup that will work but I will still to > > > try to find out how to keep logs local to the rack using the templates you > > > listed below. > > > > > > Perhaps I don't need to keep logs per rack as this seems to balance out > > > the storage pretty well. > > > What mechanism does elasticsearch use to keep the data balanced across > > > all nodes? > > > > > > 28 shards all reported on all nodes. > > > logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 > > > .. > > > logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 > > > > > > On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley [cl...@traveljury.com](mailto:cl...@traveljury.com)wrote: > > > > > > > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > > > > > > > Great, I checked out the docs and can came up with following config to > > > > > enable rack awareness for each instance that would look something like > > > > > this. > > > > > > > > > > node: > > > > > name: node1 > > > > > rack\_id: rack1 > > > > > cluster: > > > > > name: elasticsearch > > > > > routing: > > > > > allocation: > > > > > awareness: > > > > > attributes: rack\_id > > > > > > > > > > Since logstash creates an index per day(logstash-2013.02.26) per node, > > > > > would I have to do anything special to make sure that an index and > > > > > it's shards are created for each node individually? > > > > > > > > If you want 30 indices on 30 different nodes, then you need to create > > > > each index with a different name, and set the allocation on each index > > > > to tie it to a single node. > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > allocation.html[http://www.elasticsearch.org/guide/reference/index-modules/allocation.html](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > > > > > For instance, you can use index templates to say: if the index name > > > > matches "node\_1\_\*" then set index.routing.allocation.\*\*include.rack\_id > > > > to > > > > "node\_1" > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/api/admin-)\*\* > > > > indices-templates.html[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > > > > > clint > > > > > > > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > > > > receiving logs for +1k servers under low load. The goal is to have all > > > > > 30 nodes to have an index started and receiving logs for its rack to > > > > > handle high load. > > > > > > > > > > > es shards -v > > > > > > index shard pri/rep state docs size > > > > > > bytes node > > > > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > > > > 24677604861 n7 > > > > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > > > > 23641399741 n18 > > > > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > > > > 24686606451 n21 > > > > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > > > > 24695469792 n24 > > > > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > > > > 24687686161 n5 > > > > > > > > > > Any recommendations on how to configure elasticsearch rack awareness > > > > > and routing to handle this. > > > > > > > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > > > > wrote: > > > > > > > > > > ``` > > > > > > If I use the distributed system, can I force data to stay > > > > > local to the > > > > > > rack? > > > > > > > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > > > > Gormley wrote: > > > > > > Hiya > > > > > > > > > > > > OK, the picture is slowly evolving :) > > > > > > > > > > > > On Mon, 2013-02-25 at 16:23 -0800, > > > > > > > > > > ``` > > > > > > > > [beha...@gmail.com](mailto:beha...@gmail.com) > > > > > > > > > ``` > > > > > wrote: > > > > > > > So I have 30 racks at a colo and have 1 es > > > > > instance per > > > > > > rack. > > > > > > > The 1 es instance per rack is used to index all > > > > > logs for > > > > > > that rack > > > > > > > only. > > > > > > > > > > > > > > Is there anyway for each instance to join a 30 > > > > > node cluster > > > > > > as a > > > > > > > (client only) and not replicate or shard data > > > > > between them? > > > > > > > > > > > > A "client" in Elasticsearch terminology doesn't hold > > > > > any > > > > > > data. Hence > > > > > > part of the confusion. I think what you're asking > > > > > is: Can I > > > > > > have an > > > > > > index on a single node in the cluster? > > > > > > > > > > > > The answer is yes: you can create 30 indices, and > > > > > specify rack > > > > > > awareness > > > > > > for each index, so that each index sits in a single > > > > > rack. > > > > > > > > > > > > > > The purpose of the client only cluster would allow > > > > > me to > > > > > > search one > > > > > > > instance and have es query all members for data. > > > > > > > > > > > > Yes, you can connect to any node in the cluster and > > > > > query one > > > > > > or more > > > > > > indices. It will forward queries to all relevant > > > > > nodes. > > > > > > > > > > > > Note: I don't recommend this setup. Especially with > > > > > 30 nodes, > > > > > > the > > > > > > chances of one of them going down is pretty high. > > > > > Hardware > > > > > > fails. With > > > > > > your current setup (esp if you don't have any > > > > > replicas) then > > > > > > you run a > > > > > > good chance of losing data. > > > > > > > > > > > > Why not just use Elasticsearch as the distributed > > > > > system that > > > > > > it is > > > > > > intended to be? > > > > > > > > > > > > clint > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the > > > > > Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails > > > > > from it, send > > > > > > an email to elasticsearc...@googlegroups.**com. > > > > > > For more options, visit > > > > > https://groups.google.com/**groups/opt_out > > > > > > > > > > ``` > > > > > > > > . > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > > > > > ``` > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > > > . > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)\*\* > > > > topic/elasticsearch/\*\*1ZUNmPHD8wY/unsubscribe?hl=en-\*\*US[https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > > . > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > > . > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 7, 2013, 7:34am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/15 "2013-03-07T07:34:43Z")
How many days you want to keep in your cluster? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 7 mars 2013 à 07:58, Brian Harris [beharris@gmail.com](mailto:beharris@gmail.com) a écrit : > Otis > > using 28 nodes with1 daily index what would be the best shard and replication scheme? > > On Wed, Mar 6, 2013 at 3:39 AM, Otis Gospodnetic [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com) wrote: > > > Hi, > > > > 0 replicas is risky. 1 node dies and you are missing ~1/28th of your data. So repl \> 0 is definitely better. > > > > ## Otis > > > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > > > On Wednesday, March 6, 2013 1:56:53 AM UTC-5, B wrote: > > > > > Clinton, if i keep to the distributed model of a 28 node cluster with all using the same index. Would shards=14 , repl=1 be a wise choice to keep all nodes doing something? > > > > > > Right now testing shards=28 and repl=0 which is working great and the cluster is balanced with data and load but thinking long term of node failures and adding nodes in the future. > > > > > > On Feb 27, 2013 11:43 PM, "Brian Harris" [beha...@gmail.com](mailto:beha...@gmail.com) wrote: > > > > > > > ok, just playing around with 28 nodes I set shards=28 and replicas=0. > > > > > > > > I now have 28 shards all of my nodes now which is balanced. > > > > This is definitely an alternate setup that will work but I will still to try to find out how to keep logs local to the rack using the templates you listed below. > > > > > > > > Perhaps I don't need to keep logs per rack as this seems to balance out the storage pretty well. > > > > What mechanism does elasticsearch use to keep the data balanced across all nodes? > > > > > > > > 28 shards all reported on all nodes. > > > > logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 > > > > .. > > > > logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 > > > > > > > > On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley [cl...@traveljury.com](mailto:cl...@traveljury.com) wrote: > > > > > > > > > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > > > > > > > > > Great, I checked out the docs and can came up with following config to > > > > > > enable rack awareness for each instance that would look something like > > > > > > this. > > > > > > > > > > > > node: > > > > > > name: node1 > > > > > > rack\_id: rack1 > > > > > > cluster: > > > > > > name: elasticsearch > > > > > > routing: > > > > > > allocation: > > > > > > awareness: > > > > > > attributes: rack\_id > > > > > > > > > > > > Since logstash creates an index per day(logstash-2013.02.26) per node, > > > > > > would I have to do anything special to make sure that an index and > > > > > > it's shards are created for each node individually? > > > > > > > > > > If you want 30 indices on 30 different nodes, then you need to create > > > > > each index with a different name, and set the allocation on each index > > > > > to tie it to a single node. > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > > > > > > > For instance, you can use index templates to say: if the index name > > > > > matches "node\_1\_\*" then set index.routing.allocation.include.rack\_id to > > > > > "node\_1" > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > > > > > > > clint > > > > > > > > > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > > > > > receiving logs for +1k servers under low load. The goal is to have all > > > > > > 30 nodes to have an index started and receiving logs for its rack to > > > > > > handle high load. > > > > > > > > > > > > > es shards -v > > > > > > > index shard pri/rep state docs size > > > > > > > bytes node > > > > > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > > > > > 24677604861 n7 > > > > > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > > > > > 23641399741 n18 > > > > > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > > > > > 24686606451 n21 > > > > > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > > > > > 24695469792 n24 > > > > > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > > > > > 24687686161 n5 > > > > > > > > > > > > Any recommendations on how to configure elasticsearch rack awareness > > > > > > and routing to handle this. > > > > > > > > > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > > > > > wrote: > > > > > > > > > > > > ``` > > > > > > > If I use the distributed system, can I force data to stay > > > > > > local to the > > > > > > > rack? > > > > > > > > > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > > > > > Gormley wrote: > > > > > > > Hiya > > > > > > > > > > > > > > OK, the picture is slowly evolving :) > > > > > > > > > > > > > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com > > > > > > wrote: > > > > > > > > So I have 30 racks at a colo and have 1 es > > > > > > instance per > > > > > > > rack. > > > > > > > > The 1 es instance per rack is used to index all > > > > > > logs for > > > > > > > that rack > > > > > > > > only. > > > > > > > > > > > > > > > > Is there anyway for each instance to join a 30 > > > > > > node cluster > > > > > > > as a > > > > > > > > (client only) and not replicate or shard data > > > > > > between them? > > > > > > > > > > > > > > A "client" in Elasticsearch terminology doesn't hold > > > > > > any > > > > > > > data. Hence > > > > > > > part of the confusion. I think what you're asking > > > > > > is: Can I > > > > > > > have an > > > > > > > index on a single node in the cluster? > > > > > > > > > > > > > > The answer is yes: you can create 30 indices, and > > > > > > specify rack > > > > > > > awareness > > > > > > > for each index, so that each index sits in a single > > > > > > rack. > > > > > > > > > > > > > > > > The purpose of the client only cluster would allow > > > > > > me to > > > > > > > search one > > > > > > > > instance and have es query all members for data. > > > > > > > > > > > > > > Yes, you can connect to any node in the cluster and > > > > > > query one > > > > > > > or more > > > > > > > indices. It will forward queries to all relevant > > > > > > nodes. > > > > > > > > > > > > > > Note: I don't recommend this setup. Especially with > > > > > > 30 nodes, > > > > > > > the > > > > > > > chances of one of them going down is pretty high. > > > > > > Hardware > > > > > > > fails. With > > > > > > > your current setup (esp if you don't have any > > > > > > replicas) then > > > > > > > you run a > > > > > > > good chance of losing data. > > > > > > > > > > > > > > Why not just use Elasticsearch as the distributed > > > > > > system that > > > > > > > it is > > > > > > > intended to be? > > > > > > > > > > > > > > clint > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the > > > > > > Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails > > > > > > from it, send > > > > > > > an email to elasticsearc...@googlegroups.com. > > > > > > > For more options, visit > > > > > > https://groups.google.com/groups/opt_out. > > > > > > > > > > > > > > > > > > > > > > > > > > ``` > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US). > > > > > To unsubscribe from this group and all its topics, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > -- > > > > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US). > > > > > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [March 7, 2013, 1:31pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/16 "2013-03-07T13:31:55Z")
Two weeks to start. On Mar 6, 2013 11:34 PM, "David Pilato" [david@pilato.fr](mailto:david@pilato.fr) wrote: > How many days you want to keep in your cluster? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 7 mars 2013 à 07:58, Brian Harris [beharris@gmail.com](mailto:beharris@gmail.com) a écrit : > > Otis > > using 28 nodes with1 daily index what would be the best shard and > replication scheme? > > On Wed, Mar 6, 2013 at 3:39 AM, Otis Gospodnetic \< > [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com)\> wrote: > > > Hi, > > > > 0 replicas is risky. 1 node dies and you are missing ~1/28th of your > > data. So repl \> 0 is definitely better. > > > > ## Otis > > > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > > > On Wednesday, March 6, 2013 1:56:53 AM UTC-5, B wrote: > > > > > Clinton, if i keep to the distributed model of a 28 node cluster with > > > all using the same index. Would shards=14 , repl=1 be a wise choice to keep > > > all nodes doing something? > > > > > > Right now testing shards=28 and repl=0 which is working great and the > > > cluster is balanced with data and load but thinking long term of node > > > failures and adding nodes in the future. > > > On Feb 27, 2013 11:43 PM, "Brian Harris" [beha...@gmail.com](mailto:beha...@gmail.com) wrote: > > > > > > > ok, just playing around with 28 nodes I set shards=28 and replicas=0. > > > > > > > > I now have 28 shards all of my nodes now which is balanced. > > > > This is definitely an alternate setup that will work but I will still > > > > to try to find out how to keep logs local to the rack using the templates > > > > you listed below. > > > > > > > > Perhaps I don't need to keep logs per rack as this seems to balance out > > > > the storage pretty well. > > > > What mechanism does elasticsearch use to keep the data balanced across > > > > all nodes? > > > > > > > > 28 shards all reported on all nodes. > > > > logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 > > > > .. > > > > logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 > > > > > > > > On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley [cl...@traveljury.com](mailto:cl...@traveljury.com)wrote: > > > > > > > > > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > > > > > > > > > Great, I checked out the docs and can came up with following config > > > > > > to > > > > > > enable rack awareness for each instance that would look something > > > > > > like > > > > > > this. > > > > > > > > > > > > node: > > > > > > name: node1 > > > > > > rack\_id: rack1 > > > > > > cluster: > > > > > > name: elasticsearch > > > > > > routing: > > > > > > allocation: > > > > > > awareness: > > > > > > attributes: rack\_id > > > > > > > > > > > > Since logstash creates an index per day(logstash-2013.02.26) per > > > > > > node, > > > > > > would I have to do anything special to make sure that an index and > > > > > > it's shards are created for each node individually? > > > > > > > > > > If you want 30 indices on 30 different nodes, then you need to create > > > > > each index with a different name, and set the allocation on each index > > > > > to tie it to a single node. > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > > allocation.html[http://www.elasticsearch.org/guide/reference/index-modules/allocation.html](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > > > > > > > For instance, you can use index templates to say: if the index name > > > > > matches "node\_1\_\*" then set index.routing.allocation.\*\*include.rack\_id > > > > > to > > > > > "node\_1" > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/api/admin-)\*\* > > > > > indices-templates.html[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > > > > > > > clint > > > > > > > > > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > > > > > receiving logs for +1k servers under low load. The goal is to have > > > > > > all > > > > > > 30 nodes to have an index started and receiving logs for its rack to > > > > > > handle high load. > > > > > > > > > > > > > es shards -v > > > > > > > index shard pri/rep state docs size > > > > > > > bytes node > > > > > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > > > > > 24677604861 n7 > > > > > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > > > > > 23641399741 n18 > > > > > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > > > > > 24686606451 n21 > > > > > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > > > > > 24695469792 n24 > > > > > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > > > > > 24687686161 n5 > > > > > > > > > > > > Any recommendations on how to configure elasticsearch rack awareness > > > > > > and routing to handle this. > > > > > > > > > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > > > > > wrote: > > > > > > > > > > > > ``` > > > > > > > If I use the distributed system, can I force data to stay > > > > > > local to the > > > > > > > rack? > > > > > > > > > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > > > > > Gormley wrote: > > > > > > > Hiya > > > > > > > > > > > > > > OK, the picture is slowly evolving :) > > > > > > > > > > > > > > On Mon, 2013-02-25 at 16:23 -0800, > > > > > > > > > > > > ``` > > > > > > > > > > [beha...@gmail.com](mailto:beha...@gmail.com) > > > > > > > > > > > ``` > > > > > > wrote: > > > > > > > > So I have 30 racks at a colo and have 1 es > > > > > > instance per > > > > > > > rack. > > > > > > > > The 1 es instance per rack is used to index all > > > > > > logs for > > > > > > > that rack > > > > > > > > only. > > > > > > > > > > > > > > > > Is there anyway for each instance to join a 30 > > > > > > node cluster > > > > > > > as a > > > > > > > > (client only) and not replicate or shard data > > > > > > between them? > > > > > > > > > > > > > > A "client" in Elasticsearch terminology doesn't > > > > > > > > > > > > ``` > > > > > > > > > > hold > > > > > > > > > > > ``` > > > > > > any > > > > > > > data. Hence > > > > > > > part of the confusion. I think what you're asking > > > > > > is: Can I > > > > > > > have an > > > > > > > index on a single node in the cluster? > > > > > > > > > > > > > > The answer is yes: you can create 30 indices, and > > > > > > specify rack > > > > > > > awareness > > > > > > > for each index, so that each index sits in a single > > > > > > rack. > > > > > > > > > > > > > > > > The purpose of the client only cluster would > > > > > > > > > > > > ``` > > > > > > > > > > allow > > > > > > > > > > > ``` > > > > > > me to > > > > > > > search one > > > > > > > > instance and have es query all members for data. > > > > > > > > > > > > > > Yes, you can connect to any node in the cluster and > > > > > > query one > > > > > > > or more > > > > > > > indices. It will forward queries to all relevant > > > > > > nodes. > > > > > > > > > > > > > > Note: I don't recommend this setup. Especially > > > > > > > > > > > > ``` > > > > > > > > > > with > > > > > > > > > > > ``` > > > > > > 30 nodes, > > > > > > > the > > > > > > > chances of one of them going down is pretty high. > > > > > > Hardware > > > > > > > fails. With > > > > > > > your current setup (esp if you don't have any > > > > > > replicas) then > > > > > > > you run a > > > > > > > good chance of losing data. > > > > > > > > > > > > > > Why not just use Elasticsearch as the distributed > > > > > > system that > > > > > > > it is > > > > > > > intended to be? > > > > > > > > > > > > > > clint > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the > > > > > > Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails > > > > > > from it, send > > > > > > > an email to elasticsearc...@googlegroups.**com. > > > > > > > For more options, visit > > > > > > https://groups.google.com/**groups/opt_out > > > > > > > > > > > > ``` > > > > > > > > > > . > > > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > ``` > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send > > > > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > > > > . > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)\*\* > > > > > topic/elasticsearch/\*\*1ZUNmPHD8wY/unsubscribe?hl=en-\*\*US[https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > > > . > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 7, 2013, 1:48pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/17 "2013-03-07T13:48:31Z")
2 weeks = 14 days. Let's say you define one shard per day. With a replica. So you will have to store 28 shards (1 primary + 1 replica for each day). You have 28 nodes? Cool. It will fit perfectly. I think I would start with these numbers and see where it goes. Main question is: can a single shard handle your daily logs? If not, you will have to adjust the number of shards per index. And have more than one shard (or replica) on a single box. -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr | @scrutmydocs Le 7 mars 2013 à 14:31, Brian Harris [beharris@gmail.com](mailto:beharris@gmail.com) a écrit : > Two weeks to start. > > On Mar 6, 2013 11:34 PM, "David Pilato" [david@pilato.fr](mailto:david@pilato.fr) wrote: > How many days you want to keep in your cluster? > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 7 mars 2013 à 07:58, Brian Harris [beharris@gmail.com](mailto:beharris@gmail.com) a écrit : > > > Otis > > > > using 28 nodes with1 daily index what would be the best shard and replication scheme? > > > > On Wed, Mar 6, 2013 at 3:39 AM, Otis Gospodnetic [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com) wrote: > > Hi, > > > > 0 replicas is risky. 1 node dies and you are missing ~1/28th of your data. So repl \> 0 is definitely better. > > > > ## Otis > > > > ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > > > On Wednesday, March 6, 2013 1:56:53 AM UTC-5, B wrote: > > Clinton, if i keep to the distributed model of a 28 node cluster with all using the same index. Would shards=14 , repl=1 be a wise choice to keep all nodes doing something? > > > > Right now testing shards=28 and repl=0 which is working great and the cluster is balanced with data and load but thinking long term of node failures and adding nodes in the future. > > > > On Feb 27, 2013 11:43 PM, "Brian Harris" [beha...@gmail.com](mailto:beha...@gmail.com) wrote: > > ok, just playing around with 28 nodes I set shards=28 and replicas=0. > > > > I now have 28 shards all of my nodes now which is balanced. > > This is definitely an alternate setup that will work but I will still to try to find out how to keep logs local to the rack using the templates you listed below. > > > > Perhaps I don't need to keep logs per rack as this seems to balance out the storage pretty well. > > What mechanism does elasticsearch use to keep the data balanced across all nodes? > > > > 28 shards all reported on all nodes. > > logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 > > .. > > logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 > > > > On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley [cl...@traveljury.com](mailto:cl...@traveljury.com) wrote: > > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > > > Great, I checked out the docs and can came up with following config to > > > enable rack awareness for each instance that would look something like > > > this. > > > > > > node: > > > name: node1 > > > rack\_id: rack1 > > > cluster: > > > name: elasticsearch > > > routing: > > > allocation: > > > awareness: > > > attributes: rack\_id > > > > > > Since logstash creates an index per day(logstash-2013.02.26) per node, > > > would I have to do anything special to make sure that an index and > > > it's shards are created for each node individually? > > > > If you want 30 indices on 30 different nodes, then you need to create > > each index with a different name, and set the allocation on each index > > to tie it to a single node. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > For instance, you can use index templates to say: if the index name > > matches "node\_1\_\*" then set index.routing.allocation.include.rack\_id to > > "node\_1" > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > clint > > > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > > receiving logs for +1k servers under low load. The goal is to have all > > > 30 nodes to have an index started and receiving logs for its rack to > > > handle high load. > > > > > > > es shards -v > > > > index shard pri/rep state docs size > > > > bytes node > > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > > 24677604861 n7 > > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > > 23641399741 n18 > > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > > 24686606451 n21 > > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > > 24695469792 n24 > > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > > 24687686161 n5 > > > > > > Any recommendations on how to configure elasticsearch rack awareness > > > and routing to handle this. > > > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > > wrote: > > > > > > ``` > > > > If I use the distributed system, can I force data to stay > > > local to the > > > > rack? > > > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > > Gormley wrote: > > > > Hiya > > > > > > > > OK, the picture is slowly evolving :) > > > > > > > > On Mon, 2013-02-25 at 16:23 -0800, beha...@gmail.com > > > wrote: > > > > > So I have 30 racks at a colo and have 1 es > > > instance per > > > > rack. > > > > > The 1 es instance per rack is used to index all > > > logs for > > > > that rack > > > > > only. > > > > > > > > > > Is there anyway for each instance to join a 30 > > > node cluster > > > > as a > > > > > (client only) and not replicate or shard data > > > between them? > > > > > > > > A "client" in Elasticsearch terminology doesn't hold > > > any > > > > data. Hence > > > > part of the confusion. I think what you're asking > > > is: Can I > > > > have an > > > > index on a single node in the cluster? > > > > > > > > The answer is yes: you can create 30 indices, and > > > specify rack > > > > awareness > > > > for each index, so that each index sits in a single > > > rack. > > > > > > > > > > The purpose of the client only cluster would allow > > > me to > > > > search one > > > > > instance and have es query all members for data. > > > > > > > > Yes, you can connect to any node in the cluster and > > > query one > > > > or more > > > > indices. It will forward queries to all relevant > > > nodes. > > > > > > > > Note: I don't recommend this setup. Especially with > > > 30 nodes, > > > > the > > > > chances of one of them going down is pretty high. > > > Hardware > > > > fails. With > > > > your current setup (esp if you don't have any > > > replicas) then > > > > you run a > > > > good chance of losing data. > > > > > > > > Why not just use Elasticsearch as the distributed > > > system that > > > > it is > > > > intended to be? > > > > > > > > clint > > > > > > > > > > > > > > > > -- > > > > You received this message because you are subscribed to the > > > Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails > > > from it, send > > > > an email to elasticsearc...@googlegroups.com. > > > > For more options, visit > > > https://groups.google.com/groups/opt_out. > > > > > > > > > > > > > > ``` > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US). > > To unsubscribe from this group and all its topics, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US). > > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![beharris](https://avatars.discourse-cdn.com/v4/letter/b/f0a364/32.png) [@beharris](https://discuss.elastic.co/u/beharris)\ **Post date:** [March 7, 2013, 1:57pm UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/18 "2013-03-07T13:57:16Z")
Hmm, yes i have 28 nodes in different racks. A single shard cant handle the load. On Mar 7, 2013 5:48 AM, "David Pilato" [david@pilato.fr](mailto:david@pilato.fr) wrote: > 2 weeks = 14 days. > Let's say you define one shard per day. > With a replica. > > So you will have to store 28 shards (1 primary + 1 replica for each day). > You have 28 nodes? Cool. It will fit perfectly. > > I think I would start with these numbers and see where it goes. > Main question is: can a single shard handle your daily logs? > > If not, you will have to adjust the number of shards per index. And have > more than one shard (or replica) on a single box. > > -- > _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 7 mars 2013 à 14:31, Brian Harris [beharris@gmail.com](mailto:beharris@gmail.com) a écrit : > > Two weeks to start. > On Mar 6, 2013 11:34 PM, "David Pilato" [david@pilato.fr](mailto:david@pilato.fr) wrote: > > > How many days you want to keep in your cluster? > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 7 mars 2013 à 07:58, Brian Harris [beharris@gmail.com](mailto:beharris@gmail.com) a écrit : > > > > Otis > > > > using 28 nodes with1 daily index what would be the best shard and > > replication scheme? > > > > On Wed, Mar 6, 2013 at 3:39 AM, Otis Gospodnetic \< > > [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com)\> wrote: > > > > > Hi, > > > > > > 0 replicas is risky. 1 node dies and you are missing ~1/28th of your > > > data. So repl \> 0 is definitely better. > > > > > > ## Otis > > > > > > ELASTICSEARCH Performance Monitoring - > > > [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) > > > > > > On Wednesday, March 6, 2013 1:56:53 AM UTC-5, B wrote: > > > > > > > Clinton, if i keep to the distributed model of a 28 node cluster with > > > > all using the same index. Would shards=14 , repl=1 be a wise choice to keep > > > > all nodes doing something? > > > > > > > > Right now testing shards=28 and repl=0 which is working great and the > > > > cluster is balanced with data and load but thinking long term of node > > > > failures and adding nodes in the future. > > > > On Feb 27, 2013 11:43 PM, "Brian Harris" [beha...@gmail.com](mailto:beha...@gmail.com) wrote: > > > > > > > > > ok, just playing around with 28 nodes I set shards=28 and replicas=0. > > > > > > > > > > I now have 28 shards all of my nodes now which is balanced. > > > > > This is definitely an alternate setup that will work but I will still > > > > > to try to find out how to keep logs local to the rack using the templates > > > > > you listed below. > > > > > > > > > > Perhaps I don't need to keep logs per rack as this seems to balance > > > > > out the storage pretty well. > > > > > What mechanism does elasticsearch use to keep the data balanced across > > > > > all nodes? > > > > > > > > > > 28 shards all reported on all nodes. > > > > > logstash-2013.02.28 27 p STARTED 2972868 2.5gb 2753226040 node4 > > > > > .. > > > > > logstash-2013.02.28 0 p STARTED 2972863 2.5gb 2764377828 node19 > > > > > > > > > > On Wed, Feb 27, 2013 at 2:58 AM, Clinton Gormley \<[cl...@traveljury.com](mailto:cl...@traveljury.com) > > > > > > > > > > > wrote: > > > > > > > > > > > On Wed, 2013-02-27 at 00:16 -0800, B wrote: > > > > > > > > > > > > > Great, I checked out the docs and can came up with following config > > > > > > > to > > > > > > > enable rack awareness for each instance that would look something > > > > > > > like > > > > > > > this. > > > > > > > > > > > > > > node: > > > > > > > name: node1 > > > > > > > rack\_id: rack1 > > > > > > > cluster: > > > > > > > name: elasticsearch > > > > > > > routing: > > > > > > > allocation: > > > > > > > awareness: > > > > > > > attributes: rack\_id > > > > > > > > > > > > > > Since logstash creates an index per day(logstash-2013.02.26) per > > > > > > > node, > > > > > > > would I have to do anything special to make sure that an index and > > > > > > > it's shards are created for each node individually? > > > > > > > > > > > > If you want 30 indices on 30 different nodes, then you need to create > > > > > > each index with a different name, and set the allocation on each index > > > > > > to tie it to a single node. > > > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > > > allocation.html[http://www.elasticsearch.org/guide/reference/index-modules/allocation.html](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > > > > > > > > > For instance, you can use index templates to say: if the index name > > > > > > matches "node\_1\_\*" then set index.routing.allocation.\*\*include.rack\_id > > > > > > to > > > > > > "node\_1" > > > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/api/admin-)\*\* > > > > > > indices-templates.html[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > > > > > > > > > clint > > > > > > > > > > > > > Currently only 5 shards are allocated in my shard=5 rep=0 cluster > > > > > > > receiving logs for +1k servers under low load. The goal is to have > > > > > > > all > > > > > > > 30 nodes to have an index started and receiving logs for its rack to > > > > > > > handle high load. > > > > > > > > > > > > > > > es shards -v > > > > > > > > index shard pri/rep state docs size > > > > > > > > bytes node > > > > > > > > logstash-2013.02.26 0 p STARTED 28031927 22.9gb > > > > > > > > 24677604861 n7 > > > > > > > > logstash-2013.02.26 1 p STARTED 26853297 22gb > > > > > > > > 23641399741 n18 > > > > > > > > logstash-2013.02.26 2 p STARTED 28035826 22.9gb > > > > > > > > 24686606451 n21 > > > > > > > > logstash-2013.02.26 3 p STARTED 28033599 22.9gb > > > > > > > > 24695469792 n24 > > > > > > > > logstash-2013.02.26 4 p STARTED 28037600 22.9gb > > > > > > > > 24687686161 n5 > > > > > > > > > > > > > > Any recommendations on how to configure elasticsearch rack awareness > > > > > > > and routing to handle this. > > > > > > > > > > > > > > On Tuesday, February 26, 2013 10:40:36 AM UTC-8, Clinton Gormley > > > > > > > wrote: > > > > > > > > > > > > > > ``` > > > > > > > > If I use the distributed system, can I force data to stay > > > > > > > local to the > > > > > > > > rack? > > > > > > > > > > > > > > Yes. Look for rack awareness in the docs > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tuesday, February 26, 2013 1:41:46 AM UTC-8, Clinton > > > > > > > Gormley wrote: > > > > > > > > Hiya > > > > > > > > > > > > > > > > OK, the picture is slowly evolving :) > > > > > > > > > > > > > > > > On Mon, 2013-02-25 at 16:23 -0800, > > > > > > > > > > > > > > ``` > > > > > > > > > > > > [beha...@gmail.com](mailto:beha...@gmail.com) > > > > > > > > > > > > > ``` > > > > > > > wrote: > > > > > > > > > So I have 30 racks at a colo and have 1 es > > > > > > > instance per > > > > > > > > rack. > > > > > > > > > The 1 es instance per rack is used to index all > > > > > > > logs for > > > > > > > > that rack > > > > > > > > > only. > > > > > > > > > > > > > > > > > > Is there anyway for each instance to join a 30 > > > > > > > node cluster > > > > > > > > as a > > > > > > > > > (client only) and not replicate or shard data > > > > > > > between them? > > > > > > > > > > > > > > > > A "client" in Elasticsearch terminology doesn't > > > > > > > > > > > > > > ``` > > > > > > > > > > > > hold > > > > > > > > > > > > > ``` > > > > > > > any > > > > > > > > data. Hence > > > > > > > > part of the confusion. I think what you're asking > > > > > > > is: Can I > > > > > > > > have an > > > > > > > > index on a single node in the cluster? > > > > > > > > > > > > > > > > The answer is yes: you can create 30 indices, and > > > > > > > specify rack > > > > > > > > awareness > > > > > > > > for each index, so that each index sits in a > > > > > > > > > > > > > > ``` > > > > > > > > > > > > single > > > > > > > > > > > > > ``` > > > > > > > rack. > > > > > > > > > > > > > > > > > > The purpose of the client only cluster would > > > > > > > > > > > > > > ``` > > > > > > > > > > > > allow > > > > > > > > > > > > > ``` > > > > > > > me to > > > > > > > > search one > > > > > > > > > instance and have es query all members for data. > > > > > > > > > > > > > > > > Yes, you can connect to any node in the cluster > > > > > > > > > > > > > > ``` > > > > > > > > > > > > and > > > > > > > > > > > > > ``` > > > > > > > query one > > > > > > > > or more > > > > > > > > indices. It will forward queries to all relevant > > > > > > > nodes. > > > > > > > > > > > > > > > > Note: I don't recommend this setup. Especially > > > > > > > > > > > > > > ``` > > > > > > > > > > > > with > > > > > > > > > > > > > ``` > > > > > > > 30 nodes, > > > > > > > > the > > > > > > > > chances of one of them going down is pretty high. > > > > > > > Hardware > > > > > > > > fails. With > > > > > > > > your current setup (esp if you don't have any > > > > > > > replicas) then > > > > > > > > you run a > > > > > > > > good chance of losing data. > > > > > > > > > > > > > > > > Why not just use Elasticsearch as the distributed > > > > > > > system that > > > > > > > > it is > > > > > > > > intended to be? > > > > > > > > > > > > > > > > clint > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to > > > > > > > > > > > > > > ``` > > > > > > > > > > > > the > > > > > > > > > > > > > ``` > > > > > > > Google > > > > > > > > Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this group and stop receiving emails > > > > > > > from it, send > > > > > > > > an email to elasticsearc...@googlegroups.**com. > > > > > > > > For more options, visit > > > > > > > https://groups.google.com/**groups/opt_out > > > > > > > > > > > > > > ``` > > > > > > > > > > > > . > > > > > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ``` > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send > > > > > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > > > > > . > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)\*\* > > > > > > topic/elasticsearch/\*\*1ZUNmPHD8wY/unsubscribe?hl=en-\*\*US[https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > > > > . > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > > . > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/1ZUNmPHD8wY/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:47am UTC](https://discuss.elastic.co/t/elasticsearch-client-only-cluster/10869/19 "2017-07-06T02:47:45Z")