# Elasticsearch on AWS ECS - Master node itself unable to discover master **URL:** \ **Category:** Elasticsearch\ **Created:** [November 20, 2020, 9:05am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091 "2020-11-20T09:05:07Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![s0umen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s0umen/32/78903_2.png) [@s0umen](https://discuss.elastic.co/u/s0umen)\ **Post date:** [November 20, 2020, 9:05am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/1 "2020-11-20T09:05:07Z")
Hi I am trying to run elasticsearch on ECS. I am using simple elasticsearch.yml as below for the docker I am using for the Master Service. =================== Start Elasticsearch YML ==================== cluster.name: "ELKecs" node.name: mst-node-1 cluster.initial\_master\_nodes: - mst-node-1 - mst-node-2 discovery.seed\_providers: ec2 network.host: 0.0.0.0 network.publish\_host: _ec2_ transport.publish\_host: _ec2_ transport.port: 9300 http.port: 9200 discovery.ec2.tag.role: master discovery.ec2.tag.environment: dev,staging discovery.ec2.tag.ElasticSearch: "poc" discovery.ec2.endpoint: ec2.{REGION}.amazonaws.com s3.client.default.endpoint: s3.{REGION}.amazonaws.com cloud.node.auto\_attributes: true cluster.routing.allocation.awareness.attributes: aws\_availability\_zone xpack.security.enabled: false =================== End Elasticsearch YML ==================== but this results to an error, as below... and I wonder why its so. { "type": "server", "timestamp": "2020-11-20T08:48:18,089Z", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "ELKecs", "node.name": "mst-node-1", "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [mst-node-1, mst-node-2] to bootstrap a cluster: have discovered [{mst-node-1}{nEIZsF-kTEetq2duqvW3aA}{tSlxUtHSRteDzE6gekNUWg}{10.44.41.150}{10.44.41.150:9300}{cdhilmrstw}{aws\_availability\_zone=ap-southeast-1b, ml.machine\_memory=4294967296, xpack.installed=true, transform.node=true, ml.max\_open\_jobs=20}]; discovery will continue using [127.0.0.1:9300] from hosts providers and [{mst-node-1}{nEIZsF-kTEetq2duqvW3aA}{tSlxUtHSRteDzE6gekNUWg}{10.44.41.150}{10.44.41.150:9300}{cdhilmrstw}{aws\_availability\_zone=ap-southeast-1b, ml.machine\_memory=4294967296, xpack.installed=true, transform.node=true, ml.max\_open\_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0" } ---
**Author:** ![Cesar\_Garcia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_garcia/32/80129_2.png) [@Cesar\_Garcia](https://discuss.elastic.co/u/Cesar_Garcia)\ **Post date:** [December 2, 2020, 11:14pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/2 "2020-12-02T23:14:56Z")
Hi, I am having the same problem but on a different platform I am running on a private cloud provider but I think the problem is here.. > [@s0umen](#): > > have discovered [{mst-node-1}{nEIZsF-kTEetq2duqvW3aA}{tSlxUtHSRteDzE6gekNUWg}{10.44.41.150}{10.44.41.150:9300}{cdhilmrstw}{aws\_availability\_zone=ap-southeast-1b, ml.machine\_memory=4294967296, xpack.installed=true, transform.node=true, ml.max\_open\_jobs=20}] The hostname that elastic is discovering is all that and in your case and mine it should be only the hostname, but I do not know how to change that ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [December 3, 2020, 8:06am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/3 "2020-12-03T08:06:28Z")
The message says all we need, although it is quite verbose. Here's the key part: > [@s0umen](#): > > this node must discover master-eligible nodes [mst-node-1, mst-node-2] to bootstrap a cluster: have discovered [{mst-node-1}...]; So it hasn't discovered `mst-node-2`, without which it can't proceed. This is because ... > [@s0umen](#): > > discovery will continue using [127.0.0.1:9300] from hosts providers ... it's only looking on localhost. You're trying to use `discovery-ec2` but there's something wrong with its setup so it is not discovering the address of the other node. ---
**Author:** ![Cesar\_Garcia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_garcia/32/80129_2.png) [@Cesar\_Garcia](https://discuss.elastic.co/u/Cesar_Garcia)\ **Post date:** [December 3, 2020, 2:48pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/4 "2020-12-03T14:48:04Z")
Hi David, here is my error message: ```auto es02 | {"type": "server", "timestamp": "2020-12-03T14:42:57,800Z", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "es-docker-cluster", "node.name": "es02", "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [es01, es02, es03] to bootstrap a cluster: have discovered [{es02}{YJiCiKnMSnuwqWrQGF0ojg}{OPwfpbcmQuicOGrm6aR-tg}{172.20.0.2}{172.20.0.2:9300}{dilmrt}{ml.machine_memory=16643383296, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]; discovery will continue using [172.20.0.3:9300, 172.20.0.4:9300] from hosts providers and [{es02}{YJiCiKnMSnuwqWrQGF0ojg}{OPwfpbcmQuicOGrm6aR-tg}{172.20.0.2}{172.20.0.2:9300}{dilmrt}{ml.machine_memory=16643383296, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0" } ``` I am traying to use docker on centos 8 , the docker-compose file I'm using is this one: > **[Install Elasticsearch with Docker | Elasticsearch Reference \[7.10\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-compose-file)** ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [December 3, 2020, 3:03pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/5 "2020-12-03T15:03:11Z")
Same as above: it's looking for multiple nodes but only discovering one: > [@Cesar\_Garcia](#): > > `this node must discover master-eligible nodes [es01, es02, es03] to bootstrap a cluster: have discovered [{es02}{...}]; discovery will continue using [172.20.0.3:9300, 172.20.0.4:9300] from hosts providers` The addresses look reasonable here, they're not just `localhost`, so in your case I'd guess it's a connectivity issue. ---
**Author:** ![Cesar\_Garcia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_garcia/32/80129_2.png) [@Cesar\_Garcia](https://discuss.elastic.co/u/Cesar_Garcia)\ **Post date:** [December 3, 2020, 3:31pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/6 "2020-12-03T15:31:23Z")
Thanks for your reply David, ...but now I am confused, I am running this on a single node via docker, iptables has no rules and selinux is Permisive, how can this be a connectivity issue ? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [December 3, 2020, 3:42pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/7 "2020-12-03T15:42:31Z")
![](https://us1.discourse-cdn.com/elastic/original/3X/0/3/03336f11af86f52d202cba8192e0f9e0ac8c1c42.jpeg) Sorry I've no idea, it says it's looking for the other nodes at those addresses, but it's not discovering them. Perhaps there's other log messages indicating why; you could try setting `logger.org.elasticsearch.discovery: DEBUG`, perhaps that'll show some useful exceptions happening. ---
**Author:** ![Cesar\_Garcia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_garcia/32/80129_2.png) [@Cesar\_Garcia](https://discuss.elastic.co/u/Cesar_Garcia)\ **Post date:** [December 3, 2020, 3:56pm UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/8 "2020-12-03T15:56:29Z")
Ready. Disable firewalld, restart docker and done ---
**Author:** ![s0umen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s0umen/32/78903_2.png) [@s0umen](https://discuss.elastic.co/u/s0umen)\ **Post date:** [December 11, 2020, 9:38am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/9 "2020-12-11T09:38:39Z")
You are using 'firewalld' inside a docker container? or on ECS instance? Cause this service is not an enabled service in ECS enabled EC2 instance. Something else might have happened. Cause the perview of the issue in docker or kubernetes is completely different than ECS. If you found and resolved this issue in ECS platform (not Fargate) please let us know Thanks in advance ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [December 11, 2020, 11:38am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/10 "2020-12-11T11:38:45Z")
Hi @s0umen, sorry I didn't realise this was your thread originally and it got sidetracked. As I [mentioned above](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/7), I suggest setting `logger.org.elasticsearch.discovery: DEBUG` to get more visibility into the discovery process and any failures that are occurring there. If you need help understanding the resulting logs, please share them here. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 8, 2021, 11:38am UTC](https://discuss.elastic.co/t/elasticsearch-on-aws-ecs-master-node-itself-unable-to-discover-master/256091/11 "2021-01-08T11:38:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.