Need help setting up a second node

Elastic Stack Elasticsearch
21

Do not comment this out. It helps to have the node have a known name.

This should be the name of the initial master node, not the hostname. Change this to master-node once you have uncommented the node name.

If the node starts up successfully, check that it binds to the expected IP address. I rarely use ipv6 so am not sure how that is handled.

22

Quick question, did you edit the elasticsearch.yml file manually before starting up the first node?

23

Changing that didn't change anything.

@leandrojmp I'm not quite sure what you mean by that. I edited the file multiple times by hand before even starting to enroll the second node. The first node is a lab machine that has been running for quite some time now. I did restart elasticsearch multiple times during the process of enrolling the second node and I always created a new enrollment token afterwards.

24

Yeah, I asked because if I'm not wrong the enrollment-token only works if the first node has been configured using the Auto-configuration process.

But looking at the elasticsearch.yml file that you shared, it seems that it was indeed configured using the auto-configuration process.

I'm not sure what exactly is the issue here, you seem to have followed the documentation, I've never used this enrollment process as I prefer to manually configure everything.

Both configurations seems to be correct.

What do you have in the logs of both nodes?

25

In short if you use enrollment token / auto config do not touch anything

If you want to use manual then you need to set up everything matching...

from the Docs

Node name formats must match

The node names used in the cluster.initial_master_nodes list must exactly match the node.name properties of the nodes. By default the node name is set to the machine’s hostname which may or may not be fully-qualified depending on your system configuration. If each node name is a fully-qualified domain name such as master-a.example.com then you must use fully-qualified domain names in the cluster.initial_master_nodes list too; conversely if your node names are bare hostnames (without the .example.com suffix) then you must use bare hostnames in the cluster.initial_master_nodes list. If you use a mix of fully-qualified and bare hostnames, or there is some other mismatch between node.name and cluster.initial_master_nodes, then the cluster will not form successfully and you will see log messages like the following.

[master-a.example.com] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [master-a, master-b] to bootstrap a cluster: have discovered [{master-b.example.com}{...

This message shows the node names master-a.example.com and master-b.example.com as well as the cluster.initial_master_nodes entries master-a and master-b, and it is clear from this message that they do not match exactly.

Look at your error message it is speaking directly to this....

You anonymize the logs so it does not look as clear to us.

[2025-03-25T10:21:44,093][INFO ][o.e.c.c.ClusterBootstrapService] [lab-elk-1] this node has not joined a bootstrapped cluster yet; 

[cluster.initial_master_nodes] is set to [[master-node-ipv6]]
[2025-03-25T10:21:44,099][INFO ][o.e.c.c.ClusterBootstrapService] [lab-elk-1]

>>>>
skipping cluster bootstrapping as local node does not match bootstrap requirements: [[master-node-ipv6]]
>>>>>

[2025-03-25T10:21:44,913][INFO ][o.e.c.c.JoinHelper       ] [lab-elk-1] failed to join {master-node}{QMruOrUFSZS_wCuvsnT8BQ}{89tCkQwiQziGxsFOq_HuCg}
...
26

I finally found the problem!

After running the enrollment token on the second node, it creates the auto-config section in the .yml file. At the end there is this:


# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
transport.host: 0.0.0.0

Which I changed to:


# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: ["[second-node-ipv6]"]

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
transport.host: ["[second-node-ipv6]"]

I also added cluster.initial_master_nodes: ["master-node"].

After these changes, I restarted elasticsearch and the node appeared on the master node when running GET _cat/nodes?v in the dev tools.

Thanks everyone for helping me out!!