No masternode selected

Elastic Stack Elasticsearch
1

Hi, I am new to the elastic world and am currently testing the elastic stack in a non production setting.

I previously set up ELK + filebeat and got everything running (first logs from filebeat visible in Kibana via Logstash -> Elastic). Since this is part of my bachelor thesis I am now trying to replicate this setup with the documentation I wrote, but I am having issues right at the start now and I am not sure why - "I am doing everything like I did the first time", but aparently I am not.

I am using ansible to install & configure everything. I am not allowed to use the official roles & need to do it by myself for the thesis.

My test setup:
Host macBookPro '15 with i7 & 16GB RAM

Guests:
Headless Debian 9 VM via VMWare Fusion (172.16.247.134) 2 Cores 2GB RAM
root + user (user has sudo rights)
Elasticsearch
Logstash
Kibana

[Not really interesting for this question]
Headless Debian 9 VM via VMWare Fusion (172.16.247.132) 1 Core 512MB RAM
root + user (user has sudo rights)
Filebeat

Basically what I am doing is:

  • Install java 8
  • Adding the apt_key
  • Installing apt-transport-https
  • Adding the repository definition in /etc/apt/sources....
  • Updating the apt cache & install the elasticsearch package
  • Editing the elasticsearch.yml
  • Enabling the Elasticsearch service
  • Starting the elasticsearch service

If I send a curl 172.16.247.134:9200, I get following answer:

{
"name" : "ELKrebuild",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "na",
"version" : {
"number" : "7.1.1",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "7a013de",
"build_date" : "2019-05-23T14:04:00.380842Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}

If I now take a look at the elasticsearch log I think the problem is, that my single elastisearch node is not finding a master (and not electing itself as the master).
Since the log from the working setup is different from this point on from the new setup, right after "bound or publishing to a non-loopback address, enforcing bootstrap checks".
The working VM logs a cluster UUID and logs that it was elected-as-master, the non working VM logs "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and [cluster.initial_master_nodes] is empty on this node: have discovered ; discovery will continue using [127.0.0.1:9300] from hosts providers and [{ELKrebuild}{eDxm9vQURH2wVvEe4ZfnEA}{SddpOSx0Ro6RZVT5z_KDKw}{172.16.247.134}{172.16.247.134:9300}{ml.machine_memory=2082492416, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0"

I am thankfull for any directions. Thanks!

2

You should set cluster.initial_master_nodes. If you don't, the node doesn't know to form a cluster on its own, and tries to find an existing cluster to join instead.

1 Like
3

Thanks, that did the trick! I just checked, Kibana is working now as well. (I backtracked this problem from kibana not getting license information from elasticsearch and figured there are connection problems to ES, in the kibana log I found something like "kibana cant connect to the elastic master node"). If anyone is interested, David solved my "Kibana server is not ready yet" like this.

@DavidTurner Just curious now, I'm pretty sure I didnt configure cluster.initial_master_nodes the first time, but can't be 100% sure. Is it possible to start a cluster without this setting? I read something about a development mode if you dont specify seeding/initial nodes? I basically copied my *.ymls from my working setup to my new installation. Thanks again, you turned my kinda frustrating day into a good one!

4

Yes, there's an auto-bootstrapping feature which you might have unknowingly made use of with your first node. cluster.initial_master_nodes only has an effect the first time a cluster is forming; once the cluster is bootstrapped, it does nothing.

1 Like
5

Thanks again. I think I now understand why my first setup worked and the second didnt:

  • I installed & launched ELK with 100% default settings to get that working first (at this point the node elected itself as master and remembered that)
  • Later on to get access to my kibana & elasticsearch from my host macBook & the filebeat VM I added discovery.seed_hosts (so elastic runs on my external IP & not localhost)
  • I then used that config for the new setup & by setting discovery.seed_hosts right from the initialization the auto-bootstrapping got disabled and I ran into this issue

Thank you for your time and helping me further understand how things work. Cheers!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.