# How can we create an Elasticsearch cluster with AWS ECS and Docker

**URL:** <https://discuss.elastic.co/t/how-can-we-create-an-elasticsearch-cluster-with-aws-ecs-and-docker/220166>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [February 20, 2020, 11:09am UTC](https://discuss.elastic.co/t/how-can-we-create-an-elasticsearch-cluster-with-aws-ecs-and-docker/220166 "2020-02-20T11:09:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![chamiduz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chamiduz/32/63020_2.png) [@chamiduz](https://discuss.elastic.co/u/chamiduz)\
**Post date:** [February 20, 2020, 11:09am UTC](https://discuss.elastic.co/t/how-can-we-create-an-elasticsearch-cluster-with-aws-ecs-and-docker/220166/1 "2020-02-20T11:09:29Z")

</div>

Hi,

I'm trying to create an ES cluster with AWS ECS and Docker. here is what i have already done.

1. created aws infrastructure

2. ECS cluster

3. Network load balancer

4. target groups for 9200,9300 ports and attached them to the container

5. discovery.seed\_hosts can be retrieved using the load balancer because i could not find any method to update the #discovery.seed\_hosts whenever a new node is created

2020-02-20T09:52:57,445][INFO][o.e.c.c.JoinHelper] [data-2] failed to join {data-1}{YwBabAZSQOy832Hzgxg\_Lg}{Y\_s5CmoEQ5-YtYqBO736DQ}{xx.xx.xx.xx}{xx.xx.xx.xx:9300}{dim}{ml.machine\_memory=8095113216, ml.max\_open\_jobs=20, xpack.installed=true} with JoinRequest{sourceNode={data-2}{HOJG\_exiRiqfjZjd77ZH\_Q}{wJr1\_FscQ8KtXgF1v6jpVg}{192.168.0.20}{192.168.0.20:9300}{dilm}{ml.machine\_memory=8272052224, xpack.installed=true, ml.max\_open\_jobs=20}, optionalJoin=Optional[Join{term=58, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={data-2}{HOJG\_exiRiqfjZjd77ZH\_Q}{wJr1\_FscQ8KtXgF1v6jpVg}{192.168.0.20}{192.168.0.20:9300}{dilm}{ml.machine\_memory=8272052224, xpack.installed=true, ml.max\_open\_jobs=20}, targetNode={data-1}{YwBabAZSQOy832Hzgxg\_Lg}{Y\_s5CmoEQ5-YtYqBO736DQ}{xx.xx.xx.xx}{xx.xx.xx.xx:9300}{dim}{ml.machine\_memory=8095113216, ml.max\_open\_jobs=20, xpack.installed=true}}]}

xx.xx.xx.xx is the loadbancer ip

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [February 20, 2020, 12:47pm UTC](https://discuss.elastic.co/t/how-can-we-create-an-elasticsearch-cluster-with-aws-ecs-and-docker/220166/2 "2020-02-20T12:47:55Z")

</div>

You have only shared the first line of a much longer message indicating that this node `failed to join` a cluster, and the rest of the message tells us why.

> [@chamiduz](#):
>
> discovery.seed\_hosts can be retrieved using the load balancer because i could not find any method to update the #discovery.seed\_hosts whenever a new node is created

Either set `discovery.seed_hosts: dns.name.example.org` where `dns.name.example.org` resolves via DNS to the addresses of **all** the master-eligible nodes, or else use the [file-based seed hosts provider](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/modules-discovery-hosts-providers.html#file-based-hosts-provider). DNS-based discovery should play nicely with [ECS's own service discovery mechanism](https://aws.amazon.com/blogs/aws/amazon-ecs-service-discovery/).

> [@chamiduz](#):
>
> xx.xx.xx.xx is the loadbancer ip

This sounds wrong since this is also the address that `data-1` claims is its own. It does not work to hide nodes from each other using a load balancer: they must be able to connect to each other directly.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 19, 2020, 12:55pm UTC](https://discuss.elastic.co/t/how-can-we-create-an-elasticsearch-cluster-with-aws-ecs-and-docker/220166/3 "2020-03-19T12:55:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
