# Nodes not joining cluster on Centos 6.2 using ElasticSearch 5.2.2

**URL:** <https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666>\
**Category:** Elasticsearch\
**Created:** [March 30, 2017, 11:45am UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666 "2017-03-30T11:45:50Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![abhishekash](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@abhishekash](https://discuss.elastic.co/u/abhishekash)\
**Post date:** [March 30, 2017, 11:45am UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/1 "2017-03-30T11:45:50Z")

</div>

I have two nodes in different physical machines :

[cluster.name](http://cluster.name): ss  
[node.name](http://node.name): node3  
path.data: /elasticsearch/elasticsearch\_data  
path.logs: /elasticsearch/elasticsearch\_logs  
bootstrap.memory\_lock: true  
network.host: [_local_]  
discovery.zen.ping.unicast.hosts: ["30.30.30.30"]

[cluster.name](http://cluster.name): ss  
[node.name](http://node.name): node4  
path.data: /elasticsearch/elasticsearch\_data  
path.logs: /elasticsearch/elasticsearch\_logs  
bootstrap.memory\_lock: true  
network.host: [_local_]  
discovery.zen.ping.unicast.hosts: ["30.30.30.30"]

Both the machines are able to ping each other.

When i try the same from my local window boxes , they join the cluster. But, on the remote centos they are not joining.

If I change the network.host to the IP of the machine , elastic search service is not starting.

Please help. !!!

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 30, 2017, 11:58am UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/2 "2017-03-30T11:58:16Z")

</div>

You need to share some logs.

---

<div class="post-metadata">

**Author:** ![abhishekash](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@abhishekash](https://discuss.elastic.co/u/abhishekash)\
**Post date:** [March 30, 2017, 1:19pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/3 "2017-03-30T13:19:39Z")

</div>

Thanks @dadoonet .

Should i provide the ss.log which is generated for the node4 ?

Are you looking for something specific ?

Thanks  
Abhishek.

---

<div class="post-metadata">

**Author:** ![abhishekash](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@abhishekash](https://discuss.elastic.co/u/abhishekash)\
**Post date:** [March 30, 2017, 1:27pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/4 "2017-03-30T13:27:21Z")

</div>

This is log i get on the node4

[2017-03-30T09:46:18,077][INFO][o.e.n.Node] [node4] node name [node4], node ID [B0QFrXCWQEiu0Ap1FiTgtg]  
[2017-03-30T09:46:18,079][INFO][o.e.n.Node] [node4] version[5.2.2], pid[62836], build[f9d9b74/2017-02-24T17:26:45.835Z], OS[Linux/2.6.32-504.el6.x86\_64/amd64], JVM[.. 64-Bit Server VM/1.8.0\_31/25.31-b07]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [aggs-matrix-stats]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [ingest-common]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [lang-expression]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [lang-groovy]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [lang-mustache]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [lang-painless]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [percolator]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [reindex]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [transport-netty3]  
[2017-03-30T09:46:19,115][INFO][o.e.p.PluginsService] [node4] loaded module [transport-netty4]  
[2017-03-30T09:46:19,116][INFO][o.e.p.PluginsService] [node4] no plugins loaded  
[2017-03-30T09:46:21,528][INFO][o.e.n.Node] [node4] initialized  
[2017-03-30T09:46:21,528][INFO][o.e.n.Node] [node4] starting ...  
[2017-03-30T09:46:21,905][INFO][o.e.t.TransportService] [node4] publish\_address {127.0.0.1:9300}, bound\_addresses {127.0.0.1:9300}  
[2017-03-30T09:46:21,914][WARN][o.e.b.BootstrapChecks] [node4] initial heap size [17179869184] not equal to maximum heap size [34359738368]; this can cause resize pauses and prevents mlockall from locking the entire heap  
[2017-03-30T09:46:21,914][WARN][o.e.b.BootstrapChecks] [node4] memory locking requested for elasticsearch process but memory is not locked  
[2017-03-30T09:46:21,915][WARN][o.e.b.BootstrapChecks] [node4] max number of threads [1024] for user [elasticsearch] is too low, increase to at least [2048]  
[2017-03-30T09:46:21,915][WARN][o.e.b.BootstrapChecks] [node4] system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk  
[2017-03-30T09:46:25,098][INFO][o.e.c.s.ClusterService] [node4] new\_master {node4}{B0QFrXCWQEiu0Ap1FiTgtg}{Tq0PHwTtRjmrtTJbk94XeQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)  
[2017-03-30T09:46:25,153][INFO][o.e.g.GatewayService] [node4] recovered [0] indices into cluster\_state  
[2017-03-30T09:46:25,218][INFO][o.e.h.HttpServer] [node4] publish\_address {127.0.0.1:9200}, bound\_addresses {127.0.0.1:9200}  
[2017-03-30T09:46:25,218][INFO][o.e.n.Node] [node4] started  
[2017-03-30T09:46:26,588][INFO][o.e.m.j.JvmGcMonitorService] [node4] [gc][5] overhead, spent [267ms] collecting in the last [1s

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 30, 2017, 1:32pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/5 "2017-03-30T13:32:38Z")

</div>

Your server only listen to 127.0.0.1 and is not accessible from outside.

You need to change `network.host`.

> If I change the network.host to the IP of the machine , Elasticsearch service is not starting.

Yes. This is expected as I can see that:

```auto
[2017-03-30T09:46:21,914][WARN][o.e.b.BootstrapChecks] [node4] initial heap size [17179869184] not equal to maximum heap size [34359738368]; this can cause resize pauses and prevents mlockall from locking the entire heap
[2017-03-30T09:46:21,914][WARN][o.e.b.BootstrapChecks] [node4] memory locking requested for elasticsearch process but memory is not locked
[2017-03-30T09:46:21,915][WARN][o.e.b.BootstrapChecks] [node4] max number of threads [1024] for user [elasticsearch] is too low, increase to at least [2048]
[2017-03-30T09:46:21,915][WARN][o.e.b.BootstrapChecks] [node4] system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

```

When in production mode, you can't start anymore unless you fix those messages.

Please read [Bootstrap Checks | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html)

---

<div class="post-metadata">

**Author:** ![abhishekash](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@abhishekash](https://discuss.elastic.co/u/abhishekash)\
**Post date:** [March 30, 2017, 1:42pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/6 "2017-03-30T13:42:20Z")

</div>

Thanks @dadoonet for the pointers. I will try to change the configurations as per the documents .

Also wanted to know , how i am able to connect across two different windows laptops with a very simialar configuration ?

Thanks  
Abhishek,

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 30, 2017, 1:56pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/7 "2017-03-30T13:56:04Z")

</div>

> [@abhishekash](#):
>
> how i am able to connect across two different windows laptops with a very simialar configuration ?

Not sure I understand the question.

---

<div class="post-metadata">

**Author:** ![abhishekash](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@abhishekash](https://discuss.elastic.co/u/abhishekash)\
**Post date:** [April 5, 2017, 12:59pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/8 "2017-04-05T12:59:13Z")

</div>

Thanks @dadoonet for the pointers.

I have done changes in the memory lock , threads and heap changes .

But now i am getting :

[2017-04-05T12:11:48,294][DEBUG][o.e.t.n.Netty4Transport] [node-a]Bound profile [default] to address {10.33.A.B:9300}  
[2017-04-05T12:11:48,296][INFO][o.e.t.TransportService] [node-a]publish\_address {10.33.A.B:9300}, bound\_addresses {10.33.A.B:9300}  
[2017-04-05T12:11:48,302][INFO][o.e.b.BootstrapChecks] [node-a]bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks  
[2017-04-05T12:11:48,304][ERROR][o.e.b.Bootstrap] [node-a]node validation exception  
bootstrap checks failed  
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk  
[2017-04-05T12:11:48,307][INFO][o.e.n.Node] [node-a]stopping ...  
[2017-04-05T12:11:48,595][INFO][o.e.n.Node] [node-a]stopped  
[2017-04-05T12:11:48,595][INFO][o.e.n.Node] [node-a]closing ...  
[2017-04-05T12:11:48,604][TRACE][o.e.n.Node] [node-a]Close times for each service:  
StopWatch 'node\_close': running time = 0s/

if i make : transport.host: 10.33.A.B it is failing with above error

But, i make it as localhost , the loopback address its starting but obviously wont join the cluster

I am unable to get now what is the system call filter error that i should look for ?

Thanks  
Abhishek.

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [April 5, 2017, 1:01pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/9 "2017-04-05T13:01:08Z")

</div>

There are logs higher up that tell you why the system call filters failed to install. I'll tell you that on CentOS 6, the kernel support for system call filters is not there. Therefore, you should explicitly disable the system call filters: `bootstrap.system_call_filter=false`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 3, 2017, 1:01pm UTC](https://discuss.elastic.co/t/nodes-not-joining-cluster-on-centos-6-2-using-elasticsearch-5-2-2/80666/10 "2017-05-03T13:01:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
