# EC2 Discovery is not working with AutoScaling group (AWS)
**URL:** https://discuss.elastic.co/t/ec2-discovery-is-not-working-with-autoscaling-group-aws/15669
**Category:** Elasticsearch
**Created:** [February 7, 2014, 1:54pm UTC](https://discuss.elastic.co/t/ec2-discovery-is-not-working-with-autoscaling-group-aws/15669 "2014-02-07T13:54:05Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Thomas\_FATTAL](https://discuss.elastic.co/u/Thomas_FATTAL)
#### Post date: [February 7, 2014, 1:54pm UTC](https://discuss.elastic.co/t/ec2-discovery-is-not-working-with-autoscaling-group-aws/15669/1 "2014-02-07T13:54:05Z")
Hi,
I'm trying to configure two Elasticsearch nodes in AWS in the same
autoscaling group (CloudFormation).
I am having some problems with them discovering each other.
The following shows the elasticsearch.log I have on the first machine with
the instance-id "i-2db5db03".
The second machine has an instance-id "i-324e6612".
It seems that both nodes recognize each other, thanks to
discovery.ec2.tag.\* field I added but then there are some problems that
make them not to join together:
[2014-02-07 13:17:08,852][INFO][node]
[ip-10-238-225-133.ec2.internal] version[1.0.0.Beta2], pid[15342],
build[296cfbe/2013-12-02T15:46:27Z]
[2014-02-07 13:17:08,853][INFO][node]
[ip-10-238-225-133.ec2.internal] initializing ...
[2014-02-07 13:17:08,917][INFO][plugins]
[ip-10-238-225-133.ec2.internal] loaded [cloud-aws], sites [paramedic]
[2014-02-07 13:17:15,452][DEBUG][discovery.zen.ping.unicast]
[ip-10-238-225-133.ec2.internal] using initial hosts [], with
concurrent\_connects [10]
[2014-02-07 13:17:15,455][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] using ping.timeout [3s],
master\_election.filter\_client [true], master\_election.filter\_data [false]
[2014-02-07 13:17:15,456][DEBUG][discovery.zen.elect]
[ip-10-238-225-133.ec2.internal] using minimum\_master\_nodes [1]
[2014-02-07 13:17:15,457][DEBUG][discovery.zen.fd]
[ip-10-238-225-133.ec2.internal] [master] uses ping\_interval [1s],
ping\_timeout [30s], ping\_retries [3]
[2014-02-07 13:17:15,500][DEBUG][discovery.zen.fd]
[ip-10-238-225-133.ec2.internal] [node] uses ping\_interval [1s],
ping\_timeout [30s], ping\_retries [3]
[2014-02-07 13:17:16,769][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] using host\_type [PRIVATE\_IP], tags
[{platform=prod}], groups [[]] with any\_group [true], availability\_zones
[[]]
[2014-02-07 13:17:19,930][INFO][node]
[ip-10-238-225-133.ec2.internal] initialized
[2014-02-07 13:17:19,931][INFO][node]
[ip-10-238-225-133.ec2.internal] starting ...
[2014-02-07 13:17:20,455][INFO][transport]
[ip-10-238-225-133.ec2.internal] bound\_address {inet[/0.0.0.0:9300]},
publish\_address {inet[/10.238.225.133:9300]}
[2014-02-07 13:17:20,527][TRACE][discovery]
[ip-10-238-225-133.ec2.internal] waiting for 30s for the initial state to
be set by the discovery
[2014-02-07 13:17:21,981][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
[2014-02-07 13:17:21,982][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
Value:
arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
}, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
aws:cloudformation:logical-id, Value: ESASG, }, {Key:
aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
[2014-02-07 13:17:21,983][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
}, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
[2014-02-07 13:17:21,983][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] using dynamic discovery nodes []
[2014-02-07 13:17:23,744][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
[2014-02-07 13:17:23,745][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
Value:
arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
}, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
aws:cloudformation:logical-id, Value: ESASG, }, {Key:
aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
[2014-02-07 13:17:23,745][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
}, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
[2014-02-07 13:17:25,604][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] using dynamic discovery nodes []
[2014-02-07 13:17:25,604][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] full ping responses: {none}
[2014-02-07 13:17:25,605][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtered ping responses:
(filter\_client[true], filter\_data[false]) {none}
[2014-02-07 13:17:25,618][INFO][cluster.service]
[ip-10-238-225-133.ec2.internal] new\_master
[ip-10-238-225-133.ec2.internal][tUBe7JNdTquROI6UWVLNpw][inet[/10.238.225.133:9300]]{aws\_availability\_zone=us-east-1d,
max\_local\_storage\_nodes=1}, reason: zen-disco-join (elected\_as\_master)
[2014-02-07 13:17:25,752][TRACE][discovery]
[ip-10-238-225-133.ec2.internal] initial state set from discovery
[2014-02-07 13:17:25,752][INFO][discovery]
[ip-10-238-225-133.ec2.internal] elasticsearch/tUBe7JNdTquROI6UWVLNpw
[2014-02-07 13:17:25,892][INFO][http]
[ip-10-238-225-133.ec2.internal] bound\_address {inet[/0.0.0.0:9200]},
publish\_address {inet[/10.238.225.133:9200]}
[2014-02-07 13:17:25,893][INFO][node]
[ip-10-238-225-133.ec2.internal] started
[2014-02-07 13:17:25,924][INFO][gateway]
[ip-10-238-225-133.ec2.internal] recovered [0] indices into cluster\_state
I'm using Chef to configure my node with the cookbook of Karmi
([https://github.com/elasticsearch/cookbook-elasticsearch](https://github.com/elasticsearch/cookbook-elasticsearch)).
The node.json is the following:
"/etc/chef/node.json": {
"content": {
"run\_list": [
"recipe[java]",
"recipe[monit]",
"recipe[elasticsearch]",
"recipe[elasticsearch::plugins]",
"recipe[elasticsearch::aws]",
"recipe[elasticsearch::monit]"
],
"java": {
"install\_flavor": "oracle",
"oracle": {
"accept\_oracle\_download\_terms": true
},
"jdk\_version": "7"
```
},
"elasticsearch": {
"version": "1.0.0.Beta2",
"logging": {
"discovery": "TRACE"
},
"cluster": {
"name": "elasticsearch"
},
"bootstrap": {
"mlockall": false
},
"discovery": {
"type": "ec2",
"ec2": {
"groups": {"Ref": "ESSecurityGroup"},
"tag": {
"platform": "prod"
}
},
"zen": {
"ping": {
"multicast": {
"enabled": false
}
}
}
},
"cloud": {
"aws": {
"access_key": {"Ref": "ESHostKeys"},
"secret_key": {"Fn::GetAtt": [
```
"ESHostKeys", "SecretAccessKey"]},
"region": "us-east-1"
},
"ec2": {
"security\_group": {"Ref": "ESSecurityGroup"
}
}
},
"monit": {
"notify\_email": "my\_email\_address",
"mail\_format": {
"from": "from\_email\_address",
"subject": "[monit]$SERVICE$EVENTon$HOST"
,
"message": "$SERVICE$ACTION:
$DESCRIPTION"
}
},
"plugins": {
"karmi/elasticsearch-paramedic" : {}
}
}
},
"mode": "000644",
"owner": "root",
"group": "root"
}
}
I'm using Oracle 7 for ElasticSearch. The machines are Ubuntu 12.04 and I'm
using Elasticsearch 1.0.0 RC2.
It seems that the first node is seeing the second node but refuse to join
it because the tags "aws:cloudformation:stack-id", "
aws:cloudformation:stack-name", "aws:cloudformation:logical-id" and "aws:autoscaling:groupName"
are not the same. However, I checked these tags for both instances and it's
the same (as the two instances have been created in the same Autoscaling
group).
Any ideas?
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6247ec9a-e7b5-4dba-8f8d-a38fde5704bc%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6247ec9a-e7b5-4dba-8f8d-a38fde5704bc%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Thomas\_FATTAL](https://discuss.elastic.co/u/Thomas_FATTAL)
#### Post date: [February 7, 2014, 3:09pm UTC](https://discuss.elastic.co/t/ec2-discovery-is-not-working-with-autoscaling-group-aws/15669/2 "2014-02-07T15:09:57Z")
Finally, I fixed my problem.
There was a mistake for the field "discovery.ec2.groups". Instead of a
string, I had to put an array of string.
And I also forgot to add the tag "platform:prod" to CloudFormation when
launching my stack.
Fixed!
On Friday, 7 February 2014 14:54:05 UTC+1, Thomas FATTAL wrote:
> Hi,
>
> I'm trying to configure two Elasticsearch nodes in AWS in the same
> autoscaling group (CloudFormation).
> I am having some problems with them discovering each other.
>
> The following shows the elasticsearch.log I have on the first machine with
> the instance-id "i-2db5db03".
> The second machine has an instance-id "i-324e6612".
>
> It seems that both nodes recognize each other, thanks to
> discovery.ec2.tag.\* field I added but then there are some problems that
> make them not to join together:
>
> [2014-02-07 13:17:08,852][INFO][node]
> [ip-10-238-225-133.ec2.internal] version[1.0.0.Beta2], pid[15342],
> build[296cfbe/2013-12-02T15:46:27Z]
> [2014-02-07 13:17:08,853][INFO][node]
> [ip-10-238-225-133.ec2.internal] initializing ...
> [2014-02-07 13:17:08,917][INFO][plugins]
> [ip-10-238-225-133.ec2.internal] loaded [cloud-aws], sites [paramedic]
> [2014-02-07 13:17:15,452][DEBUG][discovery.zen.ping.unicast]
> [ip-10-238-225-133.ec2.internal] using initial hosts , with
> concurrent\_connects [10]
> [2014-02-07 13:17:15,455][DEBUG][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] using ping.timeout [3s],
> master\_election.filter\_client [true], master\_election.filter\_data [false]
> [2014-02-07 13:17:15,456][DEBUG][discovery.zen.elect]
> [ip-10-238-225-133.ec2.internal] using minimum\_master\_nodes [1]
> [2014-02-07 13:17:15,457][DEBUG][discovery.zen.fd]
> [ip-10-238-225-133.ec2.internal] [master] uses ping\_interval [1s],
> ping\_timeout [30s], ping\_retries [3]
> [2014-02-07 13:17:15,500][DEBUG][discovery.zen.fd]
> [ip-10-238-225-133.ec2.internal] [node] uses ping\_interval [1s],
> ping\_timeout [30s], ping\_retries [3]
> [2014-02-07 13:17:16,769][DEBUG][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] using host\_type [PRIVATE\_IP], tags
> [{platform=prod}], groups [] with any\_group [true], availability\_zones
> []
> [2014-02-07 13:17:19,930][INFO][node]
> [ip-10-238-225-133.ec2.internal] initialized
> [2014-02-07 13:17:19,931][INFO][node]
> [ip-10-238-225-133.ec2.internal] starting ...
> [2014-02-07 13:17:20,455][INFO][transport]
> [ip-10-238-225-133.ec2.internal] bound\_address {inet[/0.0.0.0:9300]},
> publish\_address {inet[/10.238.225.133:9300]}
> [2014-02-07 13:17:20,527][TRACE][discovery]
> [ip-10-238-225-133.ec2.internal] waiting for 30s for the initial state to
> be set by the discovery
> [2014-02-07 13:17:21,981][TRACE][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
> [2014-02-07 13:17:21,982][TRACE][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
> Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:cloudformation:logical-id, Value: ESASG, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:21,983][TRACE][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
> Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:21,983][DEBUG][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] using dynamic discovery nodes
> [2014-02-07 13:17:23,744][TRACE][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
> [2014-02-07 13:17:23,745][TRACE][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
> Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:cloudformation:logical-id, Value: ESASG, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:23,745][TRACE][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
> tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
> Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
> arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
> }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
> aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
> [2014-02-07 13:17:25,604][DEBUG][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] using dynamic discovery nodes
> [2014-02-07 13:17:25,604][TRACE][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] full ping responses: {none}
> [2014-02-07 13:17:25,605][DEBUG][discovery.ec2]
> [ip-10-238-225-133.ec2.internal] filtered ping responses:
> (filter\_client[true], filter\_data[false]) {none}
> [2014-02-07 13:17:25,618][INFO][cluster.service]
> [ip-10-238-225-133.ec2.internal] new\_master
> [ip-10-238-225-133.ec2.internal][tUBe7JNdTquROI6UWVLNpw][inet[/10.238.225.133:9300]]{aws\_availability\_zone=us-east-1d,
> max\_local\_storage\_nodes=1}, reason: zen-disco-join (elected\_as\_master)
> [2014-02-07 13:17:25,752][TRACE][discovery]
> [ip-10-238-225-133.ec2.internal] initial state set from discovery
> [2014-02-07 13:17:25,752][INFO][discovery]
> [ip-10-238-225-133.ec2.internal] elasticsearch/tUBe7JNdTquROI6UWVLNpw
> [2014-02-07 13:17:25,892][INFO][http]
> [ip-10-238-225-133.ec2.internal] bound\_address {inet[/0.0.0.0:9200]},
> publish\_address {inet[/10.238.225.133:9200]}
> [2014-02-07 13:17:25,893][INFO][node]
> [ip-10-238-225-133.ec2.internal] started
> [2014-02-07 13:17:25,924][INFO][gateway]
> [ip-10-238-225-133.ec2.internal] recovered [0] indices into cluster\_state
>
> I'm using Chef to configure my node with the cookbook of Karmi (
> [GitHub - sous-chefs/elasticsearch: Development repository for the elasticsearch cookbook](https://github.com/elasticsearch/cookbook-elasticsearch)).
> The node.json is the following:
>
> "/etc/chef/node.json": {
> "content": {
> "run\_list": [
> "recipe[java]",
> "recipe[monit]",
> "recipe[elasticsearch]",
> "recipe[elasticsearch::plugins]",
> "recipe[elasticsearch::aws]",
> "recipe[elasticsearch::monit]"
> ],
> "java": {
> "install\_flavor": "oracle",
> "oracle": {
> "accept\_oracle\_download\_terms": true
> },
> "jdk\_version": "7"
>
> ```
> },
> "elasticsearch": {
> "version": "1.0.0.Beta2",
> "logging": {
> "discovery": "TRACE"
> },
> "cluster": {
> "name": "elasticsearch"
> },
> "bootstrap": {
> "mlockall": false
> },
> "discovery": {
> "type": "ec2",
> "ec2": {
> "groups": {"Ref": "ESSecurityGroup"},
> "tag": {
> "platform": "prod"
> }
> },
> "zen": {
> "ping": {
> "multicast": {
> "enabled": false
> }
> }
> }
> },
> "cloud": {
> "aws": {
> "access_key": {"Ref": "ESHostKeys"},
> "secret_key": {"Fn::GetAtt": [
>
> ```
>
> "ESHostKeys", "SecretAccessKey"]},
> "region": "us-east-1"
> },
> "ec2": {
> "security\_group": {"Ref":
> "ESSecurityGroup"}
> }
> },
> "monit": {
> "notify\_email": "my\_email\_address",
> "mail\_format": {
> "from": "from\_email\_address",
> "subject":
> "[monit]$SERVICE$EVENTon$HOST",
> "message": "$SERVICE$ACTION:
> $DESCRIPTION"
> }
> },
> "plugins": {
> "karmi/elasticsearch-paramedic" : {}
> }
> }
> },
> "mode": "000644",
> "owner": "root",
> "group": "root"
> }
> }
>
> I'm using Oracle 7 for Elasticsearch. The machines are Ubuntu 12.04 and
> I'm using Elasticsearch 1.0.0 RC2.
>
> It seems that the first node is seeing the second node but refuse to join
> it because the tags "aws:cloudformation:stack-id", "
> aws:cloudformation:stack-name", "aws:cloudformation:logical-id" and "aws:autoscaling:groupName"
> are not the same. However, I checked these tags for both instances and it's
> the same (as the two instances have been created in the same Autoscaling
> group).
>
> Any ideas?
> Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/754216cd-c396-4396-a1e6-b1e6a9cc63c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/754216cd-c396-4396-a1e6-b1e6a9cc63c3%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:51am UTC](https://discuss.elastic.co/t/ec2-discovery-is-not-working-with-autoscaling-group-aws/15669/3 "2017-07-06T01:51:42Z")