# Shard awareness: multiple awareness attributes **URL:** \ **Category:** Elasticsearch\ **Created:** [November 9, 2016, 2:14pm UTC](https://discuss.elastic.co/t/shard-awareness-multiple-awareness-attributes/65506 "2016-11-09T14:14:03Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![drjz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/drjz/32/972_2.png) [@drjz](https://discuss.elastic.co/u/drjz)\ **Post date:** [November 9, 2016, 2:14pm UTC](https://discuss.elastic.co/t/shard-awareness-multiple-awareness-attributes/65506/1 "2016-11-09T14:14:03Z")
We want to make the shards awareness of the infrastructure topology. We have racks, and within those racks, we have zones. A zone consists of multiple physical machines. A physical machine has multiple virtual machines. I have been experimenting with having multiple awareness attributes, but these do not seem to be working. It is working correctly if there is 1 awareness attribute (e.g. rack). An example in the elasticsearch.yml file: > cluster.routing.allocation.awareness.attributes: rack,zone > cluster.routing.allocation.awareness.force.rack.values: rack1,rack2,rack3 > cluster.routing.allocation.awareness.force.zone.values: zone1,zone2,zone3 > node.rack: rack3 > node.zone: zone1 How can you create and use multiple awareness attributes correctly? ---
**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\ **Post date:** [November 9, 2016, 2:52pm UTC](https://discuss.elastic.co/t/shard-awareness-multiple-awareness-attributes/65506/2 "2016-11-09T14:52:33Z")
Can you provide an example on how you would expect shards to be allocated and how they are actually allocated? ---
**Author:** ![drjz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/drjz/32/972_2.png) [@drjz](https://discuss.elastic.co/u/drjz)\ **Post date:** [November 10, 2016, 6:37pm UTC](https://discuss.elastic.co/t/shard-awareness-multiple-awareness-attributes/65506/3 "2016-11-10T18:37:27Z")
I have 3 racks. On each rack, there are 5 physical machines ("zones"). On each physical machine (= 1 zone), there are virtual machines. I want to make sure the primary shards are at least in 1 rack. I want to make sure the same primary and replica shards are not in the same zone. ---
**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\ **Post date:** [November 12, 2016, 11:29am UTC](https://discuss.elastic.co/t/shard-awareness-multiple-awareness-attributes/65506/4 "2016-11-12T11:29:16Z")
Allocation awareness is not hierarchical. What's happening here is that the rack constraint ensures that shard copies are spread among the different racks. The zone constraint tries to ensure that shard copies are spread among the zones, which is however independent of the racks. Let me illustrate with an example (index with 1 primary shard and 4 replicas - only 3 racks and 2 zones for simplicity): ```auto Rack1 Rack2 Rack3 Z1 Z2 Z1 Z2 Z1 Z2 P RR RR ``` - Rack 1 has 1 shard and racks 2 and 3 have 2 shards, so the racks are balanced. - Zone 1 has 3 shards and zone 2 has 2 shards, so the zones are balanced as well. What you probably want is allocation awareness on the racks and something like `cluster.routing.allocation.same_shard.host` constraint on the zones (see here: [https://www.elastic.co/guide/en/elasticsearch/reference/5.0/shards-allocation.html#\_shard\_allocation\_settings](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/shards-allocation.html#_shard_allocation_settings)). ---
**Author:** ![drjz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/drjz/32/972_2.png) [@drjz](https://discuss.elastic.co/u/drjz)\ **Post date:** [November 14, 2016, 9:49am UTC](https://discuss.elastic.co/t/shard-awareness-multiple-awareness-attributes/65506/5 "2016-11-14T09:49:27Z")
Thank you for this clear explanation. Unfortunately `cluster.routing.allocation.same_shard.host` works only when the nodes have the same hostname AND IP address. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 12, 2016, 9:49am UTC](https://discuss.elastic.co/t/shard-awareness-multiple-awareness-attributes/65506/6 "2016-12-12T09:49:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.