# Datanodes cant see each other using EC2 Discovery Plugin **URL:** \ **Category:** Elasticsearch\ **Created:** [December 16, 2017, 1:44pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086 "2017-12-16T13:44:22Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 16, 2017, 1:44pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/1 "2017-12-16T13:44:22Z")
hello guys, im currently having an issue after configuring the elasticsearch to see the other ec2 datanodes note that im using elasticsearch-6.0.1-1 the below is my configuration ##################### [cluster.name](http://cluster.name): test-cluster [node.name](http://node.name): ${HOSTNAME} path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: [_eth0_,_local_,_ec2_] discovery.ec2.access\_key: -- discovery.ec2.secret\_key: -- discovery.ec2.availability\_zones: eu-west-1c cloud.node.auto\_attributes: true discovery.ec2.any\_group: true cluster.routing.allocation.awareness.attributes: aws\_availability\_zone ######### i put datanodes in the same subnet,security groups and zone and they are accessible to each other through 9200-9300 did i miss something? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 16, 2017, 3:30pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/2 "2017-12-16T15:30:36Z")
Could you share your logs? Also you should use the secure settings instead putting in clear your credentials. ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 16, 2017, 4:00pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/3 "2017-12-16T16:00:00Z")
thank you, im currently test the creating cluster from scratch,so i should do all running and security in future this is the log ```auto [2017-12-16T12:07:20,499][INFO][o.e.d.DiscoveryModule] [ip-10-10-227-107] using discovery type [zen] [2017-12-16T12:07:20,954][INFO][o.e.n.Node] [ip-10-10-227-107] initialized [2017-12-16T12:07:20,954][INFO][o.e.n.Node] [ip-10-10-227-107] starting ... [2017-12-16T12:07:21,101][INFO][o.e.t.TransportService] [ip-10-10-227-107] publish_address {10.10.227.107:9300}, bound_addresses {10.10.227.107:9300}, {[::1]:9300}, {127.0.0.1:9300}, {[fe80::4d0:79ff:fe62:a2fa]:9300} [2017-12-16T12:07:21,116][INFO][o.e.b.BootstrapChecks] [ip-10-10-227-107] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks [2017-12-16T12:07:25,108][INFO][o.e.c.s.MasterService] [ip-10-10-227-107] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {ip-10-10-227-107}{VzczLDl2Sqi_GoGafJA8fg}{eI97wxT5TzauC6ZVOr3QIQ}{10.10.227.107}{10.10.227.107:9300}{aws_availability_zone=eu-west-1c} [2017-12-16T12:07:25,119][INFO][o.e.c.s.ClusterApplierService] [ip-10-10-227-107] new_master {ip-10-10-227-107}{VzczLDl2Sqi_GoGafJA8fg}{eI97wxT5TzauC6ZVOr3QIQ}{10.10.227.107}{10.10.227.107:9300}{aws_availability_zone=eu-west-1c}, reason: apply cluster state (from master [master {ip-10-10-227-107}{VzczLDl2Sqi_GoGafJA8fg}{eI97wxT5TzauC6ZVOr3QIQ}{10.10.227.107}{10.10.227.107:9300}{aws_availability_zone=eu-west-1c} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]]) [2017-12-16T12:07:25,156][INFO][o.e.g.GatewayService] [ip-10-10-227-107] recovered [0] indices into cluster_state [2017-12-16T12:07:25,183][INFO][o.e.h.n.Netty4HttpServerTransport] [ip-10-10-227-107] publish_address {10.10.227.107:9200}, bound_addresses {10.10.227.107:9200}, {[::1]:9200}, {127.0.0.1:9200}, {[fe80::4d0:79ff:fe62:a2fa]:9200} [2017-12-16T12:07:25,183][INFO][o.e.n.Node] [ip-10-10-227-107] started ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 16, 2017, 8:11pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/4 "2017-12-16T20:11:04Z")
Please format your code using `` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21). It will make your post more readable. Or use markdown style like: ```` ``` CODE ``` ```` What are the other nodes logs? ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 16, 2017, 8:37pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/5 "2017-12-16T20:37:06Z")
changed:+1: i have two nodes for now,, this is the other node ```auto 12-16T12:07:15,182][INFO][o.e.d.DiscoveryModule] [ip-10-10-227-55] using discovery type [zen] [2017-12-16T12:07:15,649][INFO][o.e.n.Node] [ip-10-10-227-55] initialized [2017-12-16T12:07:15,650][INFO][o.e.n.Node] [ip-10-10-227-55] starting ... [2017-12-16T12:07:15,805][INFO][o.e.t.TransportService] [ip-10-10-227-55] publish_address {10.10.227.55:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}, {[fe80::404:f6ff:fe0d:8284]:9300}, {10.10.227.55:9300} [2017-12-16T12:07:15,816][INFO][o.e.b.BootstrapChecks] [ip-10-10-227-55] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks [2017-12-16T12:07:19,859][INFO][o.e.c.s.MasterService] [ip-10-10-227-55] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {ip-10-10-227-55}{xOQmeyPGSMK3whAd37MKXw}{WHgsFN4CS7-lwynYDPRK8Q}{10.10.227.55}{10.10.227.55:9300}{aws_availability_zone=eu-west-1c} [2017-12-16T12:07:19,866][INFO][o.e.c.s.ClusterApplierService] [ip-10-10-227-55] new_master {ip-10-10-227-55}{xOQmeyPGSMK3whAd37MKXw}{WHgsFN4CS7-lwynYDPRK8Q}{10.10.227.55}{10.10.227.55:9300}{aws_availability_zone=eu-west-1c}, reason: apply cluster state (from master [master {ip-10-10-227-55}{xOQmeyPGSMK3whAd37MKXw}{WHgsFN4CS7-lwynYDPRK8Q}{10.10.227.55}{10.10.227.55:9300}{aws_availability_zone=eu-west-1c} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]]) [2017-12-16T12:07:19,893][INFO][o.e.g.GatewayService] [ip-10-10-227-55] recovered [0] indices into cluster_state [2017-12-16T12:07:19,907][INFO][o.e.h.n.Netty4HttpServerTransport] [ip-10-10-227-55] publish_address {10.10.227.55:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}, {[fe80::404:f6ff:fe0d:8284]:9200}, {10.10.227.55:9200} [2017-12-16T12:07:19,907][INFO][o.e.n.Node] [ip-10-10-227-55] started ``` thank you ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 16, 2017, 9:43pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/6 "2017-12-16T21:43:22Z")
You need to change some settings. Please read: [https://www.elastic.co/guide/en/elasticsearch/plugins/current/discovery-ec2-usage.html](https://www.elastic.co/guide/en/elasticsearch/plugins/current/discovery-ec2-usage.html) ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 16, 2017, 10:23pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/7 "2017-12-16T22:23:40Z")
do you mean ```auto discovery.zen.hosts_provider: ec2 ``` if yes sorry i already added this to the configuration and added also ```auto discovery.ec2.host_type: private_ip ``` but with no use.maybe i will dig this issue later thanks ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 17, 2017, 4:30pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/8 "2017-12-17T16:30:14Z")
hello again, actually i went through the documentation,made some changes ,but still cant let the two nodes see each other ```auto cluster.name: dev-cluster node.name: ${HOSTNAME} path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch bootstrap.memory_lock: true network.host: [_eth0_,_local_,_ec2_] discovery.zen.hosts_provider: ec2 discovery.ec2.availability_zones: eu-west-1c cloud.node.auto_attributes: true discovery.ec2.groups: sg-xx,sg-xx discovery.ec2.any_group: true discovery.ec2.host_type: private_ip discovery.ec2.tag.Name: Elasticsearch6 cluster.routing.allocation.awareness.attributes: aws_availability_zone discovery.ec2.endpoint: eu-west-1 ``` the log stayed the same ```auto [2017-12-17T16:22:19,041][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [aggs-matrix-stats] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [analysis-common] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [ingest-common] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [lang-expression] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [lang-mustache] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [lang-painless] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [parent-join] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [percolator] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [reindex] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [repository-url] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [transport-netty4] [2017-12-17T16:22:19,042][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded module [tribe] [2017-12-17T16:22:19,049][INFO][o.e.p.PluginsService] [ip-10-10-227-107] loaded plugin [discovery-ec2] [2017-12-17T16:22:21,040][INFO][o.e.d.DiscoveryModule] [ip-10-10-227-107] using discovery type [zen] [2017-12-17T16:22:21,510][INFO][o.e.n.Node] [ip-10-10-227-107] initialized [2017-12-17T16:22:21,510][INFO][o.e.n.Node] [ip-10-10-227-107] starting ... [2017-12-17T16:22:21,660][INFO][o.e.t.TransportService] [ip-10-10-227-107] publish_address {10.10.227.107:9300}, bound_addresses {10.10.227.107:9300}, {[::1]:9300}, {127.0.0.1:9300}, {[fe80::4d0:79ff:fe62:a2fa]:9300} [2017-12-17T16:22:21,670][INFO][o.e.b.BootstrapChecks] [ip-10-10-227-107] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks [2017-12-17T16:22:21,863][INFO][o.e.d.e.AwsEc2UnicastHostsProvider] [ip-10-10-227-107] Exception while retrieving instance list from AWS API: Unable to execute HTTP request: eu-west-1: Name or service not known [2017-12-17T16:22:24,904][INFO][o.e.c.s.MasterService] [ip-10-10-227-107] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {ip-10-10-227-107}{GAri8yyrReSaR24rbU-zeQ}{8UjTAr2FT-6GpIB7oQ7Q9Q}{10.10.227.107}{10.10.227.107:9300}{aws_availability_zone=eu-west-1c} [2017-12-17T16:22:24,910][INFO][o.e.c.s.ClusterApplierService] [ip-10-10-227-107] new_master {ip-10-10-227-107}{GAri8yyrReSaR24rbU-zeQ}{8UjTAr2FT-6GpIB7oQ7Q9Q}{10.10.227.107}{10.10.227.107:9300}{aws_availability_zone=eu-west-1c}, reason: apply cluster state (from master [master {ip-10-10-227-107}{GAri8yyrReSaR24rbU-zeQ}{8UjTAr2FT-6GpIB7oQ7Q9Q}{10.10.227.107}{10.10.227.107:9300}{aws_availability_zone=eu-west-1c} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]]) [2017-12-17T16:22:24,943][INFO][o.e.g.GatewayService] [ip-10-10-227-107] recovered [0] indices into cluster_state [2017-12-17T16:22:24,950][INFO][o.e.h.n.Netty4HttpServerTransport] [ip-10-10-227-107] publish_address {10.10.227.107:9200}, bound_addresses {10.10.227.107:9200}, {[::1]:9200}, {127.0.0.1:9200}, {[fe80::4d0:79ff:fe62:a2fa]:9200} [2017-12-17T16:22:24,950][INFO][o.e.n.Node] [ip-10-10-227-107] started [2017-12-17T16:22:54,943][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:23:24,946][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:23:54,947][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:24:24,948][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:24:54,949][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:25:24,950][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:25:54,951][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:26:24,953][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [2017-12-17T16:26:48,805][TRACE][o.e.a.a.c.s.TransportClusterStateAction] [ip-10-10-227-107] Serving cluster state request using version 2 [2017-12-17T16:26:54,954][TRACE][o.e.a.a.i.s.TransportIndicesStatsAction] [ip-10-10-227-107] resolving shards for [indices:monitor/stats] based on cluster state version [2] [root@ip-10-10-227-107 elasticsearch]# ``` i dont know if there any further configuration,but i covered all the configuration for ec2 discovery,any suggestions? thank you ---
**Author:** ![mujtabahussain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mujtabahussain/32/17514_2.png) [@mujtabahussain](https://discuss.elastic.co/u/mujtabahussain)\ **Post date:** [December 17, 2017, 11:59pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/9 "2017-12-17T23:59:41Z")
> [@cloud\_sora](#): > > but with no use.maybe i will dig this issue later Can you telnet from one node to the other on ports `9200` and `9300`? ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 18, 2017, 7:56am UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/10 "2017-12-18T07:56:31Z")
yes i can telnet without issues ---
**Author:** ![mujtabahussain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mujtabahussain/32/17514_2.png) [@mujtabahussain](https://discuss.elastic.co/u/mujtabahussain)\ **Post date:** [December 18, 2017, 10:59pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/11 "2017-12-18T22:59:43Z")
Try using `_publicIp_` for discovery! ☹ ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 19, 2017, 10:02am UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/12 "2017-12-19T10:02:33Z")
@mujtabahussain it didnt work as i dont have public IP in any instance ☹ im going insane because of that 😮 ---
**Author:** ![mujtabahussain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mujtabahussain/32/17514_2.png) [@mujtabahussain](https://discuss.elastic.co/u/mujtabahussain)\ **Post date:** [December 19, 2017, 11:57pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/13 "2017-12-19T23:57:07Z")
DOH. So Sorry, I meant `_privateIp_` not public ☹ ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 20, 2017, 11:39am UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/14 "2017-12-20T11:39:23Z")
nothing worked :/. did anyone try it on the new version of elasticsearch ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 20, 2017, 1:52pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/15 "2017-12-20T13:52:12Z")
You need to change I think: ``` discovery.ec2.endpoint: eu-west-1 ``` By ``` discovery.ec2.endpoint: ec2.eu-west-1.amazonaws.com ``` ---
**Author:** ![cloud\_sora](https://avatars.discourse-cdn.com/v4/letter/c/ecccb3/32.png) [@cloud\_sora](https://discuss.elastic.co/u/cloud_sora)\ **Post date:** [December 21, 2017, 12:40pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/16 "2017-12-21T12:40:11Z")
@dadoonet u r the man,it worked 😃 😃 the configuration should be like the below ```auto cluster.name: dev-cluster node.name: ${HOSTNAME} path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch bootstrap.memory_lock: true network.host: [_eth0_,_local_,_ec2_] discovery.zen.hosts_provider: ec2 discovery.ec2.availability_zones: eu-west-1c,eu-west-1b,eu-west-1a cloud.node.auto_attributes: true #discovery.ec2.groups: sg-xx,sg-xx discovery.ec2.any_group: true discovery.ec2.host_type: private_ip discovery.ec2.tag.Name: Elasticsearch6 cluster.routing.allocation.awareness.attributes: aws_availability_zone discovery.ec2.endpoint: ec2.eu-west-1.amazonaws.com discovery.zen.minimum_master_nodes: 2 ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 18, 2018, 12:40pm UTC](https://discuss.elastic.co/t/datanodes-cant-see-each-other-using-ec2-discovery-plugin/112086/17 "2018-01-18T12:40:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.