Adding a node to a cluster

Elastic Stack Elasticsearch
1

Hi all,

I have a graylog server setup as a single host. Version are:

Host - Ubuntu 18.04
Graylog - 3.0.0-12
Java - 1.8.0_191
Elasticsearch - 6.6.2"
MongoDB - 4.0.6
Apache - 2.4.29

So I'm I want to add a node the a single cluster. I set up the new host, specs are:

Host - Ubuntu 18.04
Java - 1.8.0_191
Elasticsearch - 6.6.2"

I edited my /etc/elasticsearch/elasticsearch.yml and it looks like this:

cluster.name: graylog
node.name: crow
node.data: true

path.logs: /var/log/elasticsearch

discovery.zen.ping.unicast.hosts: ["172.24.228.161","172.24.228.226"]

.161 is my single graylog server and 226 is the new node. ES has been restarted. I have then edited my single graylog ES conf, which looks like this:

cluster.name: graylog
action.auto_create_index: false
node.master: true

node.name: grayone

path.logs: /var/log/elasticsearch

discovery.zen.ping.unicast.hosts: ["172.24.228.161","172.24.228.226"]

Config based on ES guide -

But when I go into graylog and check nodes the new one is not there. Doing a ```
curl -XGET 'http://127.0.0.1:9200/_cluster/health?pretty=true'


> {
>   "cluster_name" : "graylog",
>   "status" : "green",
>   "timed_out" : false,
>   "number_of_nodes" : 1,
>   "number_of_data_nodes" : 1,
>   "active_primary_shards" : 28,
>   "active_shards" : 28,
>   "relocating_shards" : 0,
>   "initializing_shards" : 0,
>   "unassigned_shards" : 0,
>   "delayed_unassigned_shards" : 0,
>   "number_of_pending_tasks" : 0,
>   "number_of_in_flight_fetch" : 0,
>   "task_max_waiting_in_queue_millis" : 0,
>   "active_shards_percent_as_number" : 100.0
> }


 Both servers can ping each other. There aren't any errors in the graylog.log on either host
Have I missed something in the config?
2

Most likely you missed network.host setting.

3

Thanks for the reply dadoonet.
I've added in network.host:IP on both hosts, but not getting some errors.

When running curl -XGET 'http://127.0.0.1:9200/_cluster/health?pretty=true'

I get back:

curl: (7) Failed to connect to 127.0.0.1 port 9200: Connection refused

and checking the ES logs and I'm getting

[2019-03-26T16:11:34,499][INFO ][o.e.d.z.ZenDiscovery ] [grayone] failed to send join request to master [{crow}{KQtVL-x1TbWLGcvDiGyXLA}{4_Ae58sDRty8Dwa6i-Zouw}{172.24.228.226}{172.24.228.226:9300}{ml.machine_memory=67529670656, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[crow][172.24.228.226:9300][internal:discovery/zen/join]]; nested: IllegalStateException[failure when sending a validation request to node]; nested: RemoteTransportException[[grayone][172.24.228.161:9300][internal:discovery/zen/join/validate]]; nested: IllegalArgumentException[Unknown NamedWriteable [org.elasticsearch.cluster.metadata.MetaData$Custom][licenses]]; ]

From that I can see it's failing because of "RemoteTransportException", but I've not come across it in the documentation (sorry new the ES) ?

4

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

image
image.jpg764×92 16.4 KB

Few things:

Your node is now listening on a given IP. You can't query it using 127.0.0.1. See the logs to see the http address and port.

It's failing to join the cluster because of an unknown setting or something like this. Not sure what it is.

5

Can you telnet between the hosts on ports 9200 and 9300?

6

Apologies dadoonet, Have read now.

OK so I when through the ES conf and there was no ref to 127. Checking the Graylog conf there was a 127 address in the "List of Elasticsearch hosts Graylog should connect to."

I've changed the setting to be:

Default: http://172.24.228.226:9200,http://172.24.228.161:9200

also enabled is

elasticsearch_discovery_enabled = true

This is a warning with this setting

WARNING: Automatic node discovery does not work if Elasticsearch requires authentication, e. g. with Shield

But I have not set up any authentication

Checking back in the logs on the graylog its states

[2019-03-27T08:45:00,934][INFO ][o.e.d.z.ZenDiscovery ] [grayone] failed to send join request to master [{crow}{KQtVL-x1TbWLGcvDiGyXLA}{4_Ae58sDRty8Dwa6i-Zouw}{172.24.228.226}{172.24.228.226:9300}{ml.machine_memory=67529670656, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[crow][172.24.228.226:9300][internal:discovery/zen/join]]; nested: IllegalStateException[failure when sending a validation request to node]; nested: RemoteTransportException[[grayone][172.24.228.161:9300][internal:discovery/zen/join/validate]]; nested: IllegalArgumentException[Unknown NamedWriteable [org.elasticsearch.cluster.metadata.MetaData$Custom][licenses]]; ]

From that It looks like its failing because of the validation request it trying to make, but I'm not sure what the validation is?

7

Hi mujtabahussain,

Yea I can telnet to both 9200 and 9300 on both nodes. I can nav in the browser to http://172.24.228.161:9200/ and get my ES info and when going to 9300 I get back

This is not an HTTP port

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.