Puppet elasticsearch nodes not discovering each other

[corywink]

Hello, I am using puppet to set up an elasticsearch cluster and I am having some problems getting the nodes to discover each other. I am using Puppet 4.5.2 and the elastic/elasticsearch module 5.1.1. Here is my puppet class for the elasticsearch nodes:

class profile::elasticsearch {
class { 'elasticsearch':
manage_repo => true,
repo_version => '5.x',
restart_on_change => true,
autoupgrade => true,
}

elasticsearch::instance { hiera('es_instance', 'default_instance'):
config => {
'discovery.zen.ping.unicast.hosts' => ["es-node1-hostname", "es-node1-hostname", "es-node1-hostname"],
'cluster.name' => 'build-staging',
}
}
}

I have taken out the actual hostnames of the nodes shown here but I have their fqdn in my code.

When the catalog is applied to the nodes elasticsearch is installed correctly but the nodes do not connect to one another.
When I run curl -XGET 'http://localhost:9200/_cluster/state?pretty' on each node they each show the cluster name as 'build-staging' but the cluster uuids are different and they each only show one node in the cluster. What am I doing wrong here? Am I setting the discovery hosts correctly through puppet? Thanks for the help!

[tylerjl (Tyler Langlois)]

Hi @corywink,

There's a few things to consider with your nodes' configuration:

• By default, Elasticsearch 5 will bind to localhost. You'll probably need to make changes to the network.host setting to ensure Elasticsearch is listening for all external connections.
• If you want to avoid letting the nodes come online without successfully forming a cluster, you may want to read up on the minimum_master_nodes setting (and how it helps to avoid split-brain).

I'd also check out your logfiles, which may include messages about being unable to connect to your other nodes if they are indeed not listening on an externally resolvable IP.

[corywink]

Thanks @tylerjl ,

The network.host setting is what fixed it! I just needed to set that to '0.0.0.0' to listen for all ips.