# Question shard allocation awareness? **URL:** \ **Category:** Elasticsearch\ **Created:** [September 30, 2019, 4:35am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565 "2019-09-30T04:35:45Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [September 30, 2019, 4:35am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/1 "2019-09-30T04:35:45Z")
Enabled shard allocation awareness/forced-awareness in the cluster, right now enabled for two zones `"cluster.routing.allocation.awareness.attributes" : "rack_id" "cluster.routing.allocation.awareness.force.rack_id.values" : "us-east-1a,us-east-1b"` Can I have more values for forced-awareness than enabled racks in my cluster, for example cluster with two availability zone(based on replica config), us-east-1a,us-east-1b but in future, we may split the cluster into three availability zone, can I add all the possible values of attribute `rack_id` to forced-awareness config, even though clusters are running in two racks/ availability zone `cluster.routing.allocation.awareness.force.rack_id.values" : "us-east-1a,us-east-1b,us-east-1c,us-east-1d"` something like this. I enabled it in ES test cluster, I'm not seeing any issues, but wanted to know if there are any issues because of this setup?. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 30, 2019, 6:43am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/2 "2019-09-30T06:43:49Z")
> [@CJ1](#): > > I enabled it in ES test cluster, I'm not seeing any issues, but wanted to know if there are any issues because of this setup?. It's not a great idea, but I don't think you'll see any difference if all your indices have `number_of_replicas: 1`. If you have any indices with more replicas then they will not be fully allocated and your cluster health will go `yellow`. I would recommend waiting until you have split the cluster across more zones. It is a simple change to adjust the awareness configuration via the API once you have more zones: ```auto PUT _cluster/settings { "persistent": { "cluster.routing.allocation.awareness.force.rack_id.values": "us-east-1a,us-east-1b,us-east-1c,us-east-1d" } } ``` Therefore I don't see the benefit in setting this value any earlier. ---
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [September 30, 2019, 7:05am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/3 "2019-09-30T07:05:36Z")
so I use a template for installing the elastic search, if I make it static I don't have to change it every time I deploy a new cluster and also can be deployed in any availability zones. One more question, after the shard allocation awareness in a cluster of rack 3, let's say I have one primary and 2 replicas, will the primary will be allocated in a single rack or it will be allocated in multiple racks ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 30, 2019, 7:28am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/4 "2019-09-30T07:28:40Z")
> [@CJ1](#): > > let's say I have one primary and 2 replicas, will the primary will be allocated in a single rack or it will be allocated in multiple racks I'm not sure I'm understanding the question. The primary is just another shard, each shard is allocated to one node and each node is in one rack so yes the primary will be in one rack. ---
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [September 30, 2019, 7:46am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/5 "2019-09-30T07:46:48Z")
Let me explain it in little bit detail, suppose I created an index with 5 shards and 2 replica in a shard allocation awareness enabled cluster [3 racks/3 nodes] so will it allocate primary shard in first node and replica1 in second node and replica2 in third node or it will be a mix of primary/replica in all nodes ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1aa51d4abc4fb2e3ae513682ebb96148e495760b.png) Is this an expected behaviour in shard allocation awarness enabled cluster?. thanks in advance ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [September 30, 2019, 8:07am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/6 "2019-09-30T08:07:34Z")
I see, you said "one primary" but now you are asking about the case with five primaries. Yes, they might well be spread out across the zones. ---
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [October 16, 2019, 1:10pm UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/7 "2019-10-16T13:10:36Z")
One more question , I have enabled `cluster.routing.allocation.awareness.attributes` for three racks(or three aws availability zone), and created an indices with default configuration(5 shards primary and 1 replica) How will be allocation awareness works for this case will primary and replica will be in two racks or will it split into three racks ?. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 16, 2019, 2:22pm UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/8 "2019-10-16T14:22:19Z")
Allocation awareness applies independently to each primary/replica pair. So primary and replica will be in two racks. ---
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [October 17, 2019, 7:12am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/9 "2019-10-17T07:12:32Z")
This is what, I'm trying to explain ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/d/2d116abf5d9ca332580c8f8c18e043927dbd60d8.png) , so when I try to create an index with number of replicas 1 in a cluster where allocation awareness is enabled for three racks/availabilty zones, the shard allocation is not based on racks, it is trying to allocate in all the avaiilable racks. but if the number of primary + replica equal to available racks, it works fine Is this an expected behaviour? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 17, 2019, 7:20am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/10 "2019-10-17T07:20:28Z")
I might be misinterpreting that graphic but it looks reasonable to me. Can you share the output of `GET _cat/shards` for the same situation? What exactly do you expect to be different with the allocation of `rack1`? ---
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [October 17, 2019, 7:34am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/11 "2019-10-17T07:34:30Z")
Output of `GET _cat/shards` ```auto .kibana 0 p STARTED 2 8.6kb 172.16.254.69 es-test-rack-master-b rack1 2 r STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack1 2 p STARTED 0 261b 172.16.252.222 es-test-rack-master-a rack1 1 r STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack1 1 p STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack1 3 p STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack1 3 r STARTED 0 261b 172.16.252.222 es-test-rack-master-a rack1 4 r STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack1 4 p STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack1 0 p STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack1 0 r STARTED 0 261b 172.16.252.222 es-test-rack-master-a .monitoring-es-6-2019.10.17 0 r STARTED 6783 3.6mb 172.16.251.47 es-test-rack-master-c .monitoring-es-6-2019.10.17 0 p STARTED 6783 3.4mb 172.16.252.222 es-test-rack-master-a rack2 2 p STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack2 2 r STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack2 2 r STARTED 0 261b 172.16.252.222 es-test-rack-master-a rack2 1 r STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack2 1 r STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack2 1 p STARTED 0 261b 172.16.252.222 es-test-rack-master-a rack2 3 r STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack2 3 p STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack2 3 r STARTED 0 261b 172.16.252.222 es-test-rack-master-a rack2 4 r STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack2 4 r STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack2 4 p STARTED 0 261b 172.16.252.222 es-test-rack-master-a rack2 0 r STARTED 0 261b 172.16.251.47 es-test-rack-master-c rack2 0 p STARTED 0 261b 172.16.254.69 es-test-rack-master-b rack2 0 r STARTED 0 261b 172.16.252.222 es-test-rack-master-a .monitoring-kibana-6-2019.10.17 0 p STARTED 6445 924.6kb 172.16.254.69 es-test-rack-master-b .monitoring-kibana-6-2019.10.17 0 r STARTED 6445 910.7kb 172.16.252.222 es-test-rack-master-a ``` And for `rack1` I thought it will be assigned in two racks since allocation awareness is enabled and number of replicas + primary equal to `2` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 17, 2019, 7:39am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/12 "2019-10-17T07:39:49Z")
I see. No, Elasticsearch will use all available nodes by default. If you want to restrict some indices to particular racks then you can use an [allocation filter](https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-filtering.html). ---
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [October 17, 2019, 7:48am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/13 "2019-10-17T07:48:46Z")
so the number of racks/availabilty zone should be equal to primary + replica? for example if I created a cluster in three racks/availabilty zone with allocation awarenss support, I should create indices with number of replicas 2, so that it will equally assigned on the racks so is there any recomended method when using allocation awarenss support? like any relation b/w number of replicas and racks enabled? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 17, 2019, 8:04am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/14 "2019-10-17T08:04:01Z")
No, there's not really any recommendations in this area. Primary plus one replica spread (slightly unevenly) across three racks is a common setup. ---
**Author:** ![CJ1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cj1/32/54561_2.png) [@CJ1](https://discuss.elastic.co/u/CJ1)\ **Post date:** [October 17, 2019, 8:09am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/15 "2019-10-17T08:09:00Z")
To confirm, if I have primary + one replica it's better to run in two racks or if I have primary + 2 replica then use three racks so that shards allocate evenly? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 17, 2019, 10:26am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/16 "2019-10-17T10:26:07Z")
As I said, there aren't really any recommendations in this area. It's common to have primary + one replica and three racks, which is neither of the alternatives you're asking about. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 14, 2019, 10:26am UTC](https://discuss.elastic.co/t/question-shard-allocation-awareness/201565/17 "2019-11-14T10:26:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.