I got the following Errors while trying to add more Elasticsearch nodes to my v5.2.1 Cluster with EC2 Discovery.
My solution was to replace discovery.ec2 with set the IPs with discovery.zen.ping.unicast.hosts.
Below are the errors and my elastic search.yml. Let me know if you need any other details. Thanks in advance.
ERRORS:
2017-02-24T17:53:00,799][INFO ][o.e.n.Node ] [prod_elastic_cluster_5_2-04] starting ...
[2017-02-24T17:53:00,968][WARN ][i.n.u.i.MacAddressUtil ] Failed to find a usable hardware address from the network interfaces; using random bytes: 2d:00:a6:1e:f7:f5:9e:1c
[2017-02-24T17:53:01,151][INFO ][o.e.t.TransportService ] [prod_elastic_cluster_5_2-04] publish_address {10.XXX.XXX.XXX:9300}, bound_addresses {10.XXX.XXX.XXX:9300}
[2017-02-24T17:53:01,156][INFO ][o.e.b.BootstrapChecks ] [prod_elastic_cluster_5_2-04] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-02-24T17:53:10,853][INFO ][o.e.x.m.e.Exporters ] [prod_elastic_cluster_5_2-04] skipping exporter [default_local] as it is not ready yet
[2017-02-24T17:53:20,856][INFO ][o.e.x.m.e.Exporters ] [prod_elastic_cluster_5_2-04] skipping exporter [default_local] as it is not ready yet
[2017-02-24T17:53:30,859][INFO ][o.e.x.m.e.Exporters ] [prod_elastic_cluster_5_2-04] skipping exporter [default_local] as it is not ready yet
[2017-02-24T17:53:31,211][WARN ][o.e.n.Node ] [prod_elastic_cluster_5_2-04] timed out while waiting for initial discovery state - timeout: 30s
[2017-02-24T17:53:31,218][INFO ][o.e.h.HttpServer ] [prod_elastic_cluster_5_2-04] publish_address {10.XXX.XXX.XXX:9200}, bound_addresses {10.XXX.XXX.XXX:9200}
[2017-02-24T17:53:31,219][INFO ][o.e.n.Node ] [prod_elastic_cluster_5_2-04] started
[2017-02-24T17:53:40,862][INFO ][o.e.x.m.e.Exporters ] [prod_elastic_cluster_5_2-04] skipping exporter [default_local] as it is not ready yet
[2017-02-24T17:53:50,864][INFO ][o.e.x.m.e.Exporters ] [prod_elastic_cluster_5_2-04] skipping exporter [default_local] as it is not ready yet
[2017-02-24T17:54:00,867][INFO ][o.e.x.m.e.Exporters ] [prod_elastic_cluster_5_2-04] skipping exporter [default_local] as it is not ready yet
Elasticsearch.yml Config
cluster.name: "My_Cluster"
node.name: "My_Cluster_Node"
node.master: "false"
node.data: "true"
node.ingest: "true"
bootstrap.memory_lock: true
network.host: "10.xxx.xxx.xxx"
path:
data:
- "/mnt/data/data1"
- "/mnt/data/data2"
discovery.zen.minimum_master_nodes: "2"
# Memory Limits
indices.fielddata.cache.size: 40%
indices.breaker.fielddata.limit: 60%
indices.breaker.request.limit: 40%
indices.breaker.total.limit: 70%
#script.disable_dynamic: true
http.cors.enabled: true
http.cors.allow-origin: "*"
######## EC2 Discovery ##################
cloud.aws.region: us-west-2
cloud.aws.protocol: https
cloud.aws.ec2.protocol: https
cloud.aws.read_timeout: 30s
discovery.zen.hosts_provider: ec2
discovery.ec2.any_group: "false"
discovery.ec2.groups: "My_Cluster_Sec_Group"
discovery.ec2.host_type: "private_ip"
########################################
Solution:
discovery.zen.ping.unicast.hosts:
- 10.XXX.XXX.XXX
- 10.XXX.XXX.XXX
- 10.XXX.XXX.XXX
- 10.XXX.XXX.XXX
- 10.XXX.XXX.XXX