Cluster setup using aws Ec2 discovery module

Elastic Stack Elasticsearch
1

Vm1:
cluster.name: cluster-elastic
node.name: node 1
node.master: true
node.data: false
node.ingest: false
network.host: ec2:privateIpv4
http.port: 9200
cloud.aws.access_key: *************************
cloud.aws.secret_key: ********************
cloud.aws.protocol: http
cloud.aws.ec2.protocol: http
cloud.aws.region: us-east-2
discovery.zen.hosts_provider: ec2
discovery.ec2.host_type: tag:tagname
discovery.ec2.availability_zones: "us-east-2c"
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 60s
discovery.zen.fd.ping_retries: 20

Vm2:
cluster.name: cluster-elastic
node.name: node 2
node.master: true
node.data: true
node.ingest: false
network.host: ec2:privateIpv4
http.port: 9200
cloud.aws.access_key: *************************
cloud.aws.secret_key: ********************
cloud.aws.protocol: http
cloud.aws.ec2.protocol: http
cloud.aws.region: us-east-2
discovery.zen.hosts_provider: ec2
discovery.ec2.host_type: tag:tagname
discovery.ec2.availability_zones: "us-east-2a"
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 60s
discovery.zen.fd.ping_retries: 20

Vm3:
cluster.name: cluster-elastic
node.name: node 3
node.master: true
node.data: true
node.ingest: false
network.host: ec2:privateIpv4
http.port: 9200
cloud.aws.access_key: *************************
cloud.aws.secret_key: ********************
cloud.aws.protocol: http
cloud.aws.ec2.protocol: http
cloud.aws.region: us-east-2
discovery.zen.hosts_provider: ec2
discovery.ec2.host_type: tag:tagname
discovery.ec2.availability_zones: "us-east-2a"
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 60s
discovery.zen.fd.ping_retries: 20

log:

[root@ip-172-31-34-82 elasticsearch]# tail -100 monitorpro-es.log
at org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider$DiscoNodesCache.refresh(AwsEc2UnicastHostsProvider.java:227) [discovery-ec2-5.5.0.jar:5.5.0]
at org.elasticsearch.common.util.SingleObjectCache.getOrRefresh(SingleObjectCache.java:54) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider.buildDynamicNodes(AwsEc2UnicastHostsProvider.java:102) [discovery-ec2-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.UnicastZenPing.ping(UnicastZenPing.java:309) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.UnicastZenPing.ping(UnicastZenPing.java:286) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.ZenDiscovery.pingAndWait(ZenDiscovery.java:1010) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:869) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:378) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.ZenDiscovery.access$4100(ZenDiscovery.java:83) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1188) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) [elasticsearch-5.5.0.jar:5.5.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
[2017-07-13T16:34:31,035][WARN ][o.e.d.z.ZenDiscovery ] [node-1] not enough master nodes discovered during pinging (found [[Candidate{node={node-1}{YiDgHjVITcyCJtUqtaH8_w}{2WDApO52Q-aAEbEri8PJTg}{172.31.34.82}{172.31.34.82:9300}, clusterStateVersion=-1}]], but needed [2]), pinging again
[2017-07-13T16:34:31,159][WARN ][o.e.d.e.AwsEc2UnicastHostsProvider] [node-1] failed to add i-0c031afbfc1c808bb, address node1
java.net.UnknownHostException: node1: Name or service not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:1.8.0_131]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[?:1.8.0_131]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[?:1.8.0_131]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[?:1.8.0_131]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_131]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_131]
at org.elasticsearch.transport.TcpTransport.parse(TcpTransport.java:921) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.transport.TcpTransport.addressesFromString(TcpTransport.java:876) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.transport.TransportService.addressesFromString(TransportService.java:691) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider.fetchDynamicNodes(AwsEc2UnicastHostsProvider.java:181) [discovery-ec2-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider$DiscoNodesCache.refresh(AwsEc2UnicastHostsProvider.java:242) [discovery-ec2-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider$DiscoNodesCache.refresh(AwsEc2UnicastHostsProvider.java:227) [discovery-ec2-5.5.0.jar:5.5.0]
at org.elasticsearch.common.util.SingleObjectCache.getOrRefresh(SingleObjectCache.java:54) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider.buildDynamicNodes(AwsEc2UnicastHostsProvider.java:102) [discovery-ec2-5.5.0.jar:5.5.0]
at org.elasticsearch.discovery.zen.UnicastZenPing.ping(UnicastZenPing.java:309) [elasticsearch-5131]

using ec2 dicovery module and tagname for creating a cluster and am getting this error while i try to setup my elastic search cluster kindly help me in resolving this

2

You have a space in node name, curious if that can be a reason for this issue. I would assume that maybe some uri compatible name should be used. Maybe node-1

3

i haven't given any spaces for node name

4

it is in your example above

And just to clarify - is that getting replaced by real ec2:privateIpv4? Or you keep it this way?

this log message is very clear says that the node node-1 advertises its address as node1, not the ec2:privateIpv4

5

yes actually the node names are like node-1,node-2,node-3 am just keeping (ec2:privateIpv4) it that way should change it like for eg 12.23.45.56

6

I prefer to set network.bind_host (localhost and external) and network.publish_host (only external) instead of network.host.
Curious if this is some kind of bug, that node is advertising node name, not the ip address specified in the network.host.

Also you can take a look on more verbose logging. Try to set logger.discovery.ec2 to more verbose level.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.