# Problem configuring EC2 cluster
**URL:** https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211
**Category:** Elasticsearch
**Created:** [April 3, 2012, 3:07pm UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211 "2012-04-03T15:07:50Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Ranjan\_Bagchi](https://discuss.elastic.co/u/Ranjan_Bagchi)
#### Post date: [April 3, 2012, 3:07pm UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/1 "2012-04-03T15:07:50Z")
I have to be missing something simple, but here goes. I've two ec2
instances. Ports 9300 and 9200 (and 22) are visible to each other.
Verified with nc.
I've installed elasticsearch-0.19.0 and the AWS plugin.
I've the following addition to elasticsearch.yml:
## WL additions
cluster.name: es-me
cloud:
aws:
access\_key: [ACCESS\_KEY]
secret\_key: [SECRET\_KEY]
region: us-east
discovery:
type: ec2
gateway:
type: s3
s3:
bucket: [es.me](http://es.me)
recover\_after\_time: 1m
recover\_after\_nodes: 1
expected\_nodes: 2
So what I'm seeing is the following:
Here's the initial discovery logs:
[2012-04-03 14:56:05,581][DEBUG][discovery.zen.ping.multicast] [Iron Cross]
using group [224.2.2.4], with port [54328], ttl [3], and address [null]
[2012-04-03 14:56:05,584][DEBUG][discovery.zen.ping.unicast] [Iron Cross]
using initial hosts [], with concurrent\_connects [10]
[2012-04-03 14:56:05,585][DEBUG][discovery.ec2] [Iron Cross]
using ping.timeout [3s]
[2012-04-03 14:56:05,590][DEBUG][discovery.zen.elect] [Iron Cross]
using minimum\_master\_nodes [-1]
[2012-04-03 14:56:05,591][DEBUG][discovery.zen.fd] [Iron Cross]
[master] uses ping\_interval [1s], ping\_timeout [30s], ping\_retries [3]
[2012-04-03 14:56:05,594][DEBUG][discovery.zen.fd] [Iron Cross]
[node] uses ping\_interval [1s], ping\_timeout [30s], ping\_retries [3]
Then it discovers several dozen servers on the subnet, attempts to connect
and fails (because of network security policies).
When I start my second server, it's the same thing. But they don't see
each other!!
Looking at the S3 file, it's created, but the metadata file is
just: ZV+{"meta-data":{"version":20,"templates":{}}}, so there aren't an
IP's or anything.
Help?
Thanks,
Ranjan
---
### Author:  [@Frederick\_Cheung](https://discuss.elastic.co/u/Frederick_Cheung)
#### Post date: [April 3, 2012, 9:22pm UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/2 "2012-04-03T21:22:52Z")
On Apr 3, 4:07 pm, Ranjan Bagchi [ranjan.bag...@gmail.com](mailto:ranjan.bag...@gmail.com) wrote:
> I have to be missing something simple, but here goes. I've two ec2
> instances. Ports 9300 and 9200 (and 22) are visible to each other.
> Verified with nc.
>
> I've installed elasticsearch-0.19.0 and the AWS plugin.
>
> I've the following addition to elasticsearch.yml:
>
> ## WL additions
>
> cluster.name: es-me
> cloud:
> aws:
> access\_key: [ACCESS\_KEY]
> secret\_key: [SECRET\_KEY]
> region: us-east
The region name should be us-east-1 I think
Fred
---
### Author:  [@Ranjan\_Bagchi](https://discuss.elastic.co/u/Ranjan_Bagchi)
#### Post date: [April 4, 2012, 1:01am UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/3 "2012-04-04T01:01:45Z")
Just tried it... same thing. What should there be in the s3 metadata file?
Ranjan
On Tuesday, April 3, 2012 2:22:52 PM UTC-7, Frederick Cheung wrote:
> On Apr 3, 4:07 pm, Ranjan Bagchi [ranjan.bag...@gmail.com](mailto:ranjan.bag...@gmail.com) wrote:
>
> > I have to be missing something simple, but here goes. I've two ec2
> > instances. Ports 9300 and 9200 (and 22) are visible to each other.
> > Verified with nc.
> >
> > I've installed elasticsearch-0.19.0 and the AWS plugin.
> >
> > I've the following addition to elasticsearch.yml:
> >
> > ## WL additions
> >
> > cluster.name: es-me
> > cloud:
> > aws:
> > access\_key: [ACCESS\_KEY]
> > secret\_key: [SECRET\_KEY]
> > region: us-east
>
> The region name should be us-east-1 I think
>
> Fred
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [April 4, 2012, 3:24pm UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/4 "2012-04-04T15:24:30Z")
Can you set the discovery logging to TRACE? It will also print the list of
machines it got from ec2 APIs, lets see if the second instance you startup
will get the first one. Also, it recommended to filter down by tag ro
security group the machines to lookup so they will be just the ES nodes.
Note, it is recommended to use the local gateway in general on ec2, and not
hte s3 gateway, as it comes with an overhead (as explained in the aws
tutorial).
On Wed, Apr 4, 2012 at 4:01 AM, Ranjan Bagchi [ranjan.bagchi@gmail.com](mailto:ranjan.bagchi@gmail.com)wrote:
> Just tried it... same thing. What should there be in the s3 metadata file?
>
> Ranjan
>
> On Tuesday, April 3, 2012 2:22:52 PM UTC-7, Frederick Cheung wrote:
>
> > On Apr 3, 4:07 pm, Ranjan Bagchi [ranjan.bag...@gmail.com](mailto:ranjan.bag...@gmail.com) wrote:
> >
> > > I have to be missing something simple, but here goes. I've two ec2
> > > instances. Ports 9300 and 9200 (and 22) are visible to each other.
> > > Verified with nc.
> > >
> > > I've installed elasticsearch-0.19.0 and the AWS plugin.
> > >
> > > I've the following addition to elasticsearch.yml:
> > >
> > > ## WL additions
> > >
> > > cluster.name: es-me
> > > cloud:
> > > aws:
> > > access\_key: [ACCESS\_KEY]
> > > secret\_key: [SECRET\_KEY]
> > > region: us-east
> >
> > The region name should be us-east-1 I think
> >
> > Fred
---
### Author:  [@Ranjan\_Bagchi](https://discuss.elastic.co/u/Ranjan_Bagchi)
#### Post date: [April 5, 2012, 2:18am UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/5 "2012-04-05T02:18:46Z")
Here are the two gisted logfiles:
[https://gist.github.com/2307472](https://gist.github.com/2307472)
[https://gist.github.com/2307461](https://gist.github.com/2307461)
Ranjan
On Wednesday, April 4, 2012 8:24:30 AM UTC-7, kimchy wrote:
> Can you set the discovery logging to TRACE? It will also print the list of
> machines it got from ec2 APIs, lets see if the second instance you startup
> will get the first one. Also, it recommended to filter down by tag ro
> security group the machines to lookup so they will be just the ES nodes.
>
> Note, it is recommended to use the local gateway in general on ec2, and
> not hte s3 gateway, as it comes with an overhead (as explained in the aws
> tutorial).
>
> On Wed, Apr 4, 2012 at 4:01 AM, Ranjan Bagchi [ranjan.bagchi@gmail.com](mailto:ranjan.bagchi@gmail.com)wrote:
>
> > Just tried it... same thing. What should there be in the s3 metadata
> > file?
> >
> > Ranjan
> >
> > On Tuesday, April 3, 2012 2:22:52 PM UTC-7, Frederick Cheung wrote:
> >
> > > On Apr 3, 4:07 pm, Ranjan Bagchi [ranjan.bag...@gmail.com](mailto:ranjan.bag...@gmail.com) wrote:
> > >
> > > > I have to be missing something simple, but here goes. I've two ec2
> > > > instances. Ports 9300 and 9200 (and 22) are visible to each other.
> > > > Verified with nc.
> > > >
> > > > I've installed elasticsearch-0.19.0 and the AWS plugin.
> > > >
> > > > I've the following addition to elasticsearch.yml:
> > > >
> > > > ## WL additions
> > > >
> > > > cluster.name: es-me
> > > > cloud:
> > > > aws:
> > > > access\_key: [ACCESS\_KEY]
> > > > secret\_key: [SECRET\_KEY]
> > > > region: us-east
> > >
> > > The region name should be us-east-1 I think
> > >
> > > Fred
---
### Author:  [@Ranjan\_Bagchi](https://discuss.elastic.co/u/Ranjan_Bagchi)
#### Post date: [April 5, 2012, 12:50pm UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/6 "2012-04-05T12:50:01Z")
Specifying the security group name enabled the two instances to see each
other.
So the discovery config is:
discovery:
type: ec2
ec2:
groups:
Thanks!
Ranjan
On Wednesday, April 4, 2012 7:18:46 PM UTC-7, Ranjan Bagchi wrote:
> Here are the two gisted logfiles:
> [https://gist.github.com/2307472](https://gist.github.com/2307472)
> [https://gist.github.com/2307461](https://gist.github.com/2307461)
>
> Ranjan
>
> On Wednesday, April 4, 2012 8:24:30 AM UTC-7, kimchy wrote:
>
> > Can you set the discovery logging to TRACE? It will also print the list
> > of machines it got from ec2 APIs, lets see if the second instance you
> > startup will get the first one. Also, it recommended to filter down by tag
> > ro security group the machines to lookup so they will be just the ES nodes.
> >
> > Note, it is recommended to use the local gateway in general on ec2, and
> > not hte s3 gateway, as it comes with an overhead (as explained in the aws
> > tutorial).
> >
> > On Wed, Apr 4, 2012 at 4:01 AM, Ranjan Bagchi [ranjan.bagchi@gmail.com](mailto:ranjan.bagchi@gmail.com)wrote:
> >
> > > Just tried it... same thing. What should there be in the s3 metadata
> > > file?
> > >
> > > Ranjan
> > >
> > > On Tuesday, April 3, 2012 2:22:52 PM UTC-7, Frederick Cheung wrote:
> > >
> > > > On Apr 3, 4:07 pm, Ranjan Bagchi [ranjan.bag...@gmail.com](mailto:ranjan.bag...@gmail.com) wrote:
> > > >
> > > > > I have to be missing something simple, but here goes. I've two ec2
> > > > > instances. Ports 9300 and 9200 (and 22) are visible to each other.
> > > > > Verified with nc.
> > > > >
> > > > > I've installed elasticsearch-0.19.0 and the AWS plugin.
> > > > >
> > > > > I've the following addition to elasticsearch.yml:
> > > > >
> > > > > ## WL additions
> > > > >
> > > > > cluster.name: es-me
> > > > > cloud:
> > > > > aws:
> > > > > access\_key: [ACCESS\_KEY]
> > > > > secret\_key: [SECRET\_KEY]
> > > > > region: us-east
> > > >
> > > > The region name should be us-east-1 I think
> > > >
> > > > Fred
---
### Author:  [@Hua\_Lin](https://discuss.elastic.co/u/Hua_Lin)
#### Post date: [August 28, 2012, 2:58am UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/7 "2012-08-28T02:58:02Z")
Sorry to raise this old topic. But I encounter same issue.
Near a hundred machines appear in the discovery nodes. I'm trying to use
groups to filter it, here is my setting:
discovery:
type: ec2
ec2:
ping\_timeout: 5s
groups: sgn2-stg-sss
any\_group: false
But it doesn't work, still the same issue with same log.
在 2012年4月5日星期四UTC+8下午8时50分01秒，Ranjan Bagchi写道：
> Specifying the security group name enabled the two instances to see each
> other.
>
> So the discovery config is:
> discovery:
> type: ec2
> ec2:
> groups:
>
> Thanks!
>
> Ranjan
>
> On Wednesday, April 4, 2012 7:18:46 PM UTC-7, Ranjan Bagchi wrote:
>
> > Here are the two gisted logfiles:
> > [https://gist.github.com/2307472](https://gist.github.com/2307472)
> > [https://gist.github.com/2307461](https://gist.github.com/2307461)
> >
> > Ranjan
> >
> > On Wednesday, April 4, 2012 8:24:30 AM UTC-7, kimchy wrote:
> >
> > > Can you set the discovery logging to TRACE? It will also print the list
> > > of machines it got from ec2 APIs, lets see if the second instance you
> > > startup will get the first one. Also, it recommended to filter down by tag
> > > ro security group the machines to lookup so they will be just the ES nodes.
> > >
> > > Note, it is recommended to use the local gateway in general on ec2, and
> > > not hte s3 gateway, as it comes with an overhead (as explained in the aws
> > > tutorial).
> > >
> > > On Wed, Apr 4, 2012 at 4:01 AM, Ranjan Bagchi \<[ranjan...@gmail.com](mailto:ranjan...@gmail.com)\
> > >
> > > > wrote:
> > >
> > > > Just tried it... same thing. What should there be in the s3 metadata
> > > > file?
> > > >
> > > > Ranjan
> > > >
> > > > On Tuesday, April 3, 2012 2:22:52 PM UTC-7, Frederick Cheung wrote:
> > > >
> > > > > On Apr 3, 4:07 pm, Ranjan Bagchi [ranjan.bag...@gmail.com](mailto:ranjan.bag...@gmail.com) wrote:
> > > > >
> > > > > > I have to be missing something simple, but here goes. I've two ec2
> > > > > > instances. Ports 9300 and 9200 (and 22) are visible to each other.
> > > > > > Verified with nc.
> > > > > >
> > > > > > I've installed elasticsearch-0.19.0 and the AWS plugin.
> > > > > >
> > > > > > I've the following addition to elasticsearch.yml:
> > > > > >
> > > > > > ## WL additions
> > > > > >
> > > > > > cluster.name: es-me
> > > > > > cloud:
> > > > > > aws:
> > > > > > access\_key: [ACCESS\_KEY]
> > > > > > secret\_key: [SECRET\_KEY]
> > > > > > region: us-east
> > > > >
> > > > > The region name should be us-east-1 I think
> > > > >
> > > > > Fred
--
---
### Author:  [@Hua\_Lin](https://discuss.elastic.co/u/Hua_Lin)
#### Post date: [August 28, 2012, 3:07am UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/8 "2012-08-28T03:07:16Z")
I encounter same issue. About 98 machines to ping, most of them don't have
ES installed.
Don't know how to set groups filter, here is my config:
discovery:
type: ec2
ec2:
ping\_timeout: 5s
groups: sgn2-stg-sss
any\_group: false
But it doesn't work. Still same log with same issue.
在 2012年4月5日星期四UTC+8下午8时50分01秒，Ranjan Bagchi写道：
> Specifying the security group name enabled the two instances to see each
> other.
>
> So the discovery config is:
> discovery:
> type: ec2
> ec2:
> groups:
>
> Thanks!
>
> Ranjan
>
> On Wednesday, April 4, 2012 7:18:46 PM UTC-7, Ranjan Bagchi wrote:
>
> > Here are the two gisted logfiles:
> > [https://gist.github.com/2307472](https://gist.github.com/2307472)
> > [https://gist.github.com/2307461](https://gist.github.com/2307461)
> >
> > Ranjan
> >
> > On Wednesday, April 4, 2012 8:24:30 AM UTC-7, kimchy wrote:
> >
> > > Can you set the discovery logging to TRACE? It will also print the list
> > > of machines it got from ec2 APIs, lets see if the second instance you
> > > startup will get the first one. Also, it recommended to filter down by tag
> > > ro security group the machines to lookup so they will be just the ES nodes.
> > >
> > > Note, it is recommended to use the local gateway in general on ec2, and
> > > not hte s3 gateway, as it comes with an overhead (as explained in the aws
> > > tutorial).
> > >
> > > On Wed, Apr 4, 2012 at 4:01 AM, Ranjan Bagchi \<[ranjan...@gmail.com](mailto:ranjan...@gmail.com)\
> > >
> > > > wrote:
> > >
> > > > Just tried it... same thing. What should there be in the s3 metadata
> > > > file?
> > > >
> > > > Ranjan
> > > >
> > > > On Tuesday, April 3, 2012 2:22:52 PM UTC-7, Frederick Cheung wrote:
> > > >
> > > > > On Apr 3, 4:07 pm, Ranjan Bagchi [ranjan.bag...@gmail.com](mailto:ranjan.bag...@gmail.com) wrote:
> > > > >
> > > > > > I have to be missing something simple, but here goes. I've two ec2
> > > > > > instances. Ports 9300 and 9200 (and 22) are visible to each other.
> > > > > > Verified with nc.
> > > > > >
> > > > > > I've installed elasticsearch-0.19.0 and the AWS plugin.
> > > > > >
> > > > > > I've the following addition to elasticsearch.yml:
> > > > > >
> > > > > > ## WL additions
> > > > > >
> > > > > > cluster.name: es-me
> > > > > > cloud:
> > > > > > aws:
> > > > > > access\_key: [ACCESS\_KEY]
> > > > > > secret\_key: [SECRET\_KEY]
> > > > > > region: us-east
> > > > >
> > > > > The region name should be us-east-1 I think
> > > > >
> > > > > Fred
--
---
### Author:  [@Hua\_Lin](https://discuss.elastic.co/u/Hua_Lin)
#### Post date: [August 28, 2012, 3:10am UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/9 "2012-08-28T03:10:31Z")
I encounter same issue. 98 machines need to ping, most of them doesn't have
ES installed.
Here is my setting:
discovery:
type: ec2
ec2:
ping\_timeout: 5s
groups: sgn2-stg-sss
any\_group: false
But it doesn't work, how to filter them out?
--
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:14am UTC](https://discuss.elastic.co/t/problem-configuring-ec2-cluster/7211/10 "2017-07-06T03:14:52Z")