Where to place unicast_hosts.txt file for cluster setup in ES 6.5

Elastic Stack Elasticsearch
1

Where to place unicast_hosts.txt file for cluster setup in ES 6.5?

In elasticsearch website (6.5) it is mentioned to put the unicast_hosts.txt file in below directory:

$ES_PATH_CONF/discovery-file/unicast_hosts.txt

I have installed ES 6.5

[root@xxxx discovery-file]# pwd
/usr/share/elasticsearch/plugins/discovery-file
[root@xxxx discovery-file]# ls -ltr
total 28
-rw-r--r--. 1 root root 1503 May 20 16:21 plugin-descriptor.properties
-rw-r--r--. 1 root root 152 May 20 16:21 NOTICE.txt
-rw-r--r--. 1 root root 6368 May 20 16:21 discovery-file-6.5.4.jar
-rw-r--r--. 1 root root 11358 May 20 16:21 LICENSE.txt

Now do I need to put the unicast_hosts.txt inside discovery-file-6.5.4.jar or just in this directory.

Please suggest!

2

No. It should be in the elasticsearch config dir. The same directory as you have elasticsearch.yml.

So if your config dir is: /foo, then place this file under /foo/discovery-file/unicast_hosts.txt.

depending on how you installed elasticsearch, it might be in /etc/elasticsearch/discovery-file/unicast_hosts.txt.

3

Oh okay. Got it. But one doubt,

In our project elasticsearch.yml file present in /etc/elasticsearch

-rw-r-xr-x 1 root space 2656 Oct 9 2018 java.policy
-rw-r-xr-x 1 root space 12422 Apr 16 16:10 log4j2.properties
drwxr-x--- 2 root space 4096 May 11 03:34 scripts
-rw-r-xr-x 1 root space 3196 May 11 03:34 jvm.options
-rw-rw---- 1 elasticsearch space 207 May 11 03:35 elasticsearch.keystore
-rw-r-xr-x 1 elasticsearch space 1016 May 20 11:20 elasticsearch.yml

So we have to create a sub-directory discovery-file under /etc/elasticsearch and then we have to keep the unicast_hosts.xt file over here. Am I correct?

4

Correct.

5

I have done like this but still no luck...

[root@xxxxxx elasticsearch]# pwd
/etc/elasticsearch
[root@xxxxxx elasticsearch]# ls -ltr
total 40
-rw-r-xr-x 1 root space 2656 Oct 9 2018 java.policy
-rw-r-xr-x 1 root space 12422 Apr 16 16:10 log4j2.properties
drwxr-x--- 2 root space 4096 May 9 08:29 scripts
-rw-r-xr-x 1 root space 3196 May 9 08:29 jvm.options
-rw-rw---- 1 elasticsearch space 207 May 9 08:33 elasticsearch.keystore
-rw-r-xr-x 1 elasticsearch space 1016 May 20 11:19 elasticsearch.yml
drwxr-xr-x 2 root root 4096 May 20 13:53 discovery-file
[root@xxxxxx elasticsearch]# cd discovery-file
[root@xxxxxx discovery-file]# ls -ltr
total 4
-rwxr-xr-x 1 root root 1023 May 20 13:53 unicast_hosts.txt

I have mentioned the node ips with 9300 port in unicast_hosts.txt file but still cluster is not formed. But cluster is formed if we put node ips in discovery.zen.ping.unicast.hosts in elasticsearch.yml file...

We need to use unicast_hosts.txt only..Please help.

6

What is your elasticsearch.yml looking like when you add the discovery-file configuration?
What are elasticsearch logs?

I wonder if discovery-file dir and files should not belong to elasticsearch user instead BTW.

7

This is elasticsearch.yml content in one of the node...is anything I missed?

cluster.name: log****
path.data: /var/cache/secmgt-elastic-data/log****
path.logs: /var/log/elasticsearch
node.master: true
node.data: true
bootstrap.memory_lock: true # this one added new
#discovery.zen.ping.unicast.hosts: ["127.0.0.1:9302", "127.0.0.1:9303"]
#discovery.zen.ping.unicast.hosts: ["10.X.X.250:9300", "10.X.X.249:9300"]
#set the master node IP if the master is remote

#Configuration to check out of memory error
indices.fielddata.cache.size:  40%
indices.breaker.fielddata.limit: 60%
indices.breaker.request.limit: 40%
indices.breaker.total.limit: 70%
#Configuration to enable performance Writes
#index.translog.flush_threshold_size: 1gb //need to set
#index.refresh_interval: 10s //need to set
#set the following only on SSD, default is good for spin disk
#indices.store.throttle.max_bytes_per_sec: 100mb
bootstrap.system_call_filter: false
network.host: 0.0.0.0
network.publish_host: _eth0_

------ END ------

I have disabled discovery.zen.ping.unicast.hosts so that the host details should be read from unicast_hosts.txt not directly from yml file.

discovery-file dir and files have been changed to elasticsearch user still issue persists, previously it was root user.

Please let me know if anything needs to be added in elasticsearch.yml file to form the cluster by reading unicast_hosts.txt?

8

Hmmm. I think that you need to add:

discovery.type: file

But I can't find this in the documentation :smirk:
Could you try and tell me if this works?

9

No Man..Elasticsearch is not starting after I have added that in yml file.

Error logs:

java.lang.IllegalArgumentException: Unknown discovery type [file]
at org.elasticsearch.discovery.DiscoveryModule.(DiscoveryModule.java:134) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.node.Node.(Node.java:529) ~[elasticsearch-6.5.4.jar:6.5.4]

Is unicast_hosts.txt based cluster formation really works in ES 6.5.4 ?

10

Ok. Thanks for confirming. So this is not needed and should work out of the box.

Could you share your elasticsearch logs?

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

image
image.jpg764×92 16.4 KB

There's a live preview panel for exactly this reasons.

11

The issue is resolved. :slight_smile:

I have added this below entry in yml file and it worked.

discovery.zen.hosts_provider: file

You were correct but I think you just did a typo

discovery.type: file

Thank you very much for the help!

2 Likes
12

Another thing :slight_smile:

Do I need to have the unicast_hosts.txt file with the node IPs only on the master node OR I need to have the text file in all the other nodes as well?

13

All nodes need to know where to find a master node.

14

Okay so need to keep the unicast_hosts.txt file in all the nodes....

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.