# Shard Allocation Awareness in ec2 **URL:** \ **Category:** Elasticsearch\ **Created:** [December 14, 2011, 6:26am UTC](https://discuss.elastic.co/t/shard-allocation-awareness-in-ec2/6156 "2011-12-14T06:26:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Piavlo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/piavlo/32/1214_2.png) [@Piavlo](https://discuss.elastic.co/u/Piavlo)\ **Post date:** [December 14, 2011, 6:26am UTC](https://discuss.elastic.co/t/shard-allocation-awareness-in-ec2/6156/1 "2011-12-14T06:26:14Z")
Hi, I'm setting up elastic search cluster in ec2 This is the discovery configuration ```auto discovery: type: ec2 ec2: groups: elasticsearch host_type: private_ip availability_zones: us-east-1a,us-east-1c,us- east-1d any_group: false ping_timeout: 5 tag: stage: dev ``` Now From the documentation it's not clear if Cluster module allocation awareness makes use of the fact that node is running in ec2 - which seems natural to do. So the question is do I need to set routing.allocation.awareness explicitly like this: cluster: name: dev routing.allocation.awareness: attributes: availability\_zone force.availability\_zone.values: us-east-1a,us- east-1c,us-east-1d node: availability\_zone: us-east-1a Or this could be simplified or completely omitted with same effect? Like specifying node.availability\_zone - seems to be unneeded - as we already know such info from the ec2 discovery module. Thanks Alex ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 14, 2011, 3:54pm UTC](https://discuss.elastic.co/t/shard-allocation-awareness-in-ec2/6156/2 "2011-12-14T15:54:56Z")
Yes, you will need to explicitly set the awareness in the routing allocation forced part. You can set cloud.node.auto\_attributes to true and it will automatically add the ec2 relevant (node.aws\_availability\_zone) to the node attributes. On Wed, Dec 14, 2011 at 8:26 AM, Piavlo [lolitushka@gmail.com](mailto:lolitushka@gmail.com) wrote: > Hi, > > I'm setting up Elasticsearch cluster in ec2 > > This is the discovery configuration > > discovery: > type: ec2 > ec2: > groups: elasticsearch > host\_type: private\_ip > availability\_zones: us-east-1a,us-east-1c,us- > east-1d > any\_group: false > ping\_timeout: 5 > tag: > stage: dev > > Now > > From the documentation it's not clear if Cluster module allocation > awareness makes use of the fact that node is running in ec2 - which > seems natural to do. > > So the question is do I need to set routing.allocation.awareness > explicitly like this: > > cluster: > name: dev > routing.allocation.awareness: > attributes: > availability\_zone > force.availability\_zone.values: us-east-1a,us- > east-1c,us-east-1d > > node: > availability\_zone: us-east-1a > > Or this could be simplified or completely omitted with same effect? > Like specifying node.availability\_zone - seems to be unneeded - as we > already know such info from the ec2 discovery module. > > Thanks > Alex ---
**Author:** ![Piavlo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/piavlo/32/1214_2.png) [@Piavlo](https://discuss.elastic.co/u/Piavlo)\ **Post date:** [December 14, 2011, 6:15pm UTC](https://discuss.elastic.co/t/shard-allocation-awareness-in-ec2/6156/3 "2011-12-14T18:15:55Z")
Hi Shay Thanks for the rpley To be sure I understood you correctly - could you please confirm that the following is a full correct config to do the job? * * * cluster: name: dev routing.allocation.awareness: attributes: aws\_availability\_zone force.aws\_availability\_zone.values: us-east-1a,us- east-1c,us-east-1d cloud: aws: access\_key: xxxxxxxxxxxxxxxxx secret\_key: xxxxxxxxxxxxxxxxx region: us-east-1 node: auto\_attributes: true ## discovery: type: ec2 ec2: groups: elasticsearch host\_type: private\_ip availability\_zones: us-east-1a,us-east-1c,us- east-1d any\_group: false ping\_timeout: 5 tag: stage: dev On Dec 14, 5:54 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > Yes, you will need to explicitly set the awareness in the routing > allocation forced part. You can set cloud.node.auto\_attributes to true and > it will automatically add the ec2 relevant (node.aws\_availability\_zone) to > the node attributes. > > On Wed, Dec 14, 2011 at 8:26 AM, Piavlo [lolitus...@gmail.com](mailto:lolitus...@gmail.com) wrote: > > > Hi, > > > I'm setting up Elasticsearch cluster in ec2 > > > This is the discovery configuration > > > discovery: > > type: ec2 > > ec2: > > groups: elasticsearch > > host\_type: private\_ip > > availability\_zones: us-east-1a,us-east-1c,us- > > east-1d > > any\_group: false > > ping\_timeout: 5 > > tag: > > stage: dev > > > Now > > > From the documentation it's not clear if Cluster module allocation > > awareness makes use of the fact that node is running in ec2 - which > > seems natural to do. > > > So the question is do I need to set routing.allocation.awareness > > explicitly like this: > > > cluster: > > name: dev > > routing.allocation.awareness: > > attributes: > > availability\_zone > > force.availability\_zone.values: us-east-1a,us- > > east-1c,us-east-1d > > > node: > > availability\_zone: us-east-1a > > > Or this could be simplified or completely omitted with same effect? > > Like specifying node.availability\_zone - seems to be unneeded - as we > > already know such info from the ec2 discovery module. > > > Thanks > > Alex ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 14, 2011, 10:10pm UTC](https://discuss.elastic.co/t/shard-allocation-awareness-in-ec2/6156/4 "2011-12-14T22:10:06Z")
It looks good, though it is hard to verify the structure of it because you pasted it in a mail (gist is better). One thing that I would verify is that the correct node attributes for the availability zones are set, that should be simple to see with node info API. On Wed, Dec 14, 2011 at 8:15 PM, Piavlo [lolitushka@gmail.com](mailto:lolitushka@gmail.com) wrote: > Hi Shay > > Thanks for the rpley > > To be sure I understood you correctly - could you please confirm that > the following is a full correct config to do the job? > > * * * > > cluster: > name: dev > routing.allocation.awareness: > attributes: > aws\_availability\_zone > force.aws\_availability\_zone.values: us-east-1a,us- > east-1c,us-east-1d > > cloud: > aws: > access\_key: xxxxxxxxxxxxxxxxx > secret\_key: xxxxxxxxxxxxxxxxx > region: us-east-1 > node: > auto\_attributes: true > > ## discovery: type: ec2 ec2: groups: elasticsearch host\_type: private\_ip availability\_zones: us-east-1a,us-east-1c,us- east-1d any\_group: false ping\_timeout: 5 tag: stage: dev > > On Dec 14, 5:54 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > Yes, you will need to explicitly set the awareness in the routing > > allocation forced part. You can set cloud.node.auto\_attributes to true > > and > > it will automatically add the ec2 relevant (node.aws\_availability\_zone) > > to > > the node attributes. > > > > On Wed, Dec 14, 2011 at 8:26 AM, Piavlo [lolitus...@gmail.com](mailto:lolitus...@gmail.com) wrote: > > > > > Hi, > > > > > I'm setting up Elasticsearch cluster in ec2 > > > > > This is the discovery configuration > > > > > discovery: > > > type: ec2 > > > ec2: > > > groups: elasticsearch > > > host\_type: private\_ip > > > availability\_zones: us-east-1a,us-east-1c,us- > > > east-1d > > > any\_group: false > > > ping\_timeout: 5 > > > tag: > > > stage: dev > > > > > Now > > > > > From the documentation it's not clear if Cluster module allocation > > > awareness makes use of the fact that node is running in ec2 - which > > > seems natural to do. > > > > > So the question is do I need to set routing.allocation.awareness > > > explicitly like this: > > > > > cluster: > > > name: dev > > > routing.allocation.awareness: > > > attributes: > > > availability\_zone > > > force.availability\_zone.values: us-east-1a,us- > > > east-1c,us-east-1d > > > > > node: > > > availability\_zone: us-east-1a > > > > > Or this could be simplified or completely omitted with same effect? > > > Like specifying node.availability\_zone - seems to be unneeded - as we > > > already know such info from the ec2 discovery module. > > > > > Thanks > > > Alex ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:45am UTC](https://discuss.elastic.co/t/shard-allocation-awareness-in-ec2/6156/5 "2017-07-06T03:45:23Z")