# Elasticsearch Transport Client fails to recovery connection after cluster restart

**URL:** <https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042>\
**Category:** Elasticsearch\
**Created:** [May 9, 2017, 8:49am UTC](https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042 "2017-05-09T08:49:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![merlob](https://avatars.discourse-cdn.com/v4/letter/m/a183cd/32.png) [@merlob](https://discuss.elastic.co/u/merlob)\
**Post date:** [May 9, 2017, 8:49am UTC](https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042/1 "2017-05-09T08:49:58Z")

</div>

Hi,

I'm using Elasticsearch 5.4.0 with Transport Client with the following problems:

- you must launch the application several times before it can connect to the cluster.

- when connection is established, after a cluster restart, the connection is no more recovered, with this stack:

The Transport Client is setting as following:

```
            Settings.Builder settingsBuilder = Settings.builder();

		settingsBuilder.put("cluster.name", "globevmes5");
		settingsBuilder.put("client.transport.sniff", true);
		 	
		client = new PreBuiltTransportClient(settingsBuilder.build());
		try {
			client.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("192.168.203.128"), 9500));
			
		} catch (Exception e) {
			System.out.println(e.getMessage());	
		}

```

and Elasticsearch tcp port is 9500.

With previous elasticsearch 5.3.2 it worked fine.

Any ideas to solve it?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 9, 2017, 9:13am UTC](https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042/2 "2017-05-09T09:13:00Z")

</div>

Can you share the logs of the elasticsearch node?

---

<div class="post-metadata">

**Author:** ![merlob](https://avatars.discourse-cdn.com/v4/letter/m/a183cd/32.png) [@merlob](https://discuss.elastic.co/u/merlob)\
**Post date:** [May 9, 2017, 9:20am UTC](https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042/3 "2017-05-09T09:20:30Z")

</div>

```
[2017-05-09T11:14:24,336][WARN][o.e.b.Natives] unable to load JNA native support library, native methods will be disabled.
java.lang.UnsatisfiedLinkError: /tmp/jna--1077556979/jna7634687564598757394.tmp: /lib64/libc.so.6: version `GLIBC_2.7' not found (required by /tmp/jna--1077556979/jna7634687564598757394.tmp)
        at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[?:1.8.0_102]
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[?:1.8.0_102]
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[?:1.8.0_102]
        at java.lang.Runtime.load0(Runtime.java:809) ~[?:1.8.0_102]
        at java.lang.System.load(System.java:1086) ~[?:1.8.0_102]
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947) ~[jna-4.4.0.jar:4.4.0 (b0)]
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) ~[jna-4.4.0.jar:4.4.0 (b0)]
        at com.sun.jna.Native.<clinit>(Native.java:190) ~[jna-4.4.0.jar:4.4.0 (b0)]
        at java.lang.Class.forName0(Native Method) ~[?:1.8.0_102]
        at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_102]
        at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:45) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:105) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:204) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.4.0.jar:5.4.0]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.4.0.jar:5.4.0]
[2017-05-09T11:14:24,342][WARN][o.e.b.Natives] cannot check if running as root because JNA is not available
[2017-05-09T11:14:24,342][WARN][o.e.b.Natives] cannot register console handler because JNA is not available
[2017-05-09T11:14:24,344][WARN][o.e.b.Natives] cannot getrlimit RLIMIT_NPROC because JNA is not available
[2017-05-09T11:14:24,344][WARN][o.e.b.Natives] cannot getrlimit RLIMIT_AS beacuse JNA is not available
[2017-05-09T11:14:24,493][INFO][o.e.n.Node] [globevmes5-node] initializing ...
[2017-05-09T11:14:24,615][INFO][o.e.e.NodeEnvironment] [globevmes5-node] using [1] data paths, mounts [[/methode (/dev/mapper/VolGroup01-LogVol02)]], net usable_space [8.1gb], net total_space [72.8gb], spins? [possibly], types [ext3]
[2017-05-09T11:14:24,615][INFO][o.e.e.NodeEnvironment] [globevmes5-node] heap size [1007.3mb], compressed ordinary object pointers [true]
[2017-05-09T11:14:24,659][INFO][o.e.n.Node] [globevmes5-node] node name [globevmes5-node], node ID [M1_iHcSKRX6wHkD_Va0uDg]
[2017-05-09T11:14:24,659][INFO][o.e.n.Node] [globevmes5-node] version[5.4.0], pid[31204], build[780f8c4/2017-04-28T17:43:27.229Z], OS[Linux/2.6.18-194.el5/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_102/25.102-b14]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [aggs-matrix-stats]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [ingest-common]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [lang-expression]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [lang-groovy]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [lang-mustache]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [lang-painless]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [percolator]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [reindex]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [transport-netty3]
[2017-05-09T11:14:26,981][INFO][o.e.p.PluginsService] [globevmes5-node] loaded module [transport-netty4]
[2017-05-09T11:14:26,982][INFO][o.e.p.PluginsService] [globevmes5-node] loaded plugin [eom-elasticsearch-plugin]
[2017-05-09T11:14:28,406][INFO][o.e.d.DiscoveryModule] [globevmes5-node] using discovery type [zen]
[2017-05-09T11:14:28,972][INFO][o.e.n.Node] [globevmes5-node] initialized
[2017-05-09T11:14:28,972][INFO][o.e.n.Node] [globevmes5-node] starting ...
[2017-05-09T11:14:29,100][INFO][o.e.t.TransportService] [globevmes5-node] publish_address {192.168.203.128:9500}, bound_addresses {192.168.203.128:9500}
[2017-05-09T11:14:29,106][INFO][o.e.b.BootstrapChecks] [globevmes5-node] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-05-09T11:14:32,227][INFO][o.e.c.s.ClusterService] [globevmes5-node] new_master {globevmes5-node}{M1_iHcSKRX6wHkD_Va0uDg}{j8GR_IGFSfW2y502jd-SKA}{192.168.203.128}{192.168.203.128:9500}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-05-09T11:14:32,379][INFO][o.e.h.n.Netty4HttpServerTransport] [globevmes5-node] publish_address {192.168.203.128:9400}, bound_addresses {192.168.203.128:9400}
[2017-05-09T11:14:32,386][INFO][o.e.n.Node] [globevmes5-node] started
[2017-05-09T11:14:32,590][INFO][o.e.g.GatewayService] [globevmes5-node] recovered [6] indices into cluster_state
[2017-05-09T11:14:33,173][INFO][o.e.c.r.a.AllocationService] [globevmes5-node] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.kibana][0]] ...]).
[2017-05-09T11:15:02,357][INFO][o.e.c.r.a.DiskThresholdMonitor] [globevmes5-node] low disk watermark [85%] exceeded on [M1_iHcSKRX6wHkD_Va0uDg][globevmes5-node][/methode/meth01/mnt/elasticsearch-5.4.0/data/nodes/0] free: 8.1gb[11.1%], replicas will not be assigned to this node

```

Setting "client.transport.sniff" to false works fine.

thx

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 9, 2017, 9:34am UTC](https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042/4 "2017-05-09T09:34:06Z")

</div>

> [@merlob](#):
>
> Setting "client.transport.sniff" to false works fine.

May be open an issue then?

---

<div class="post-metadata">

**Author:** ![billnbell](https://avatars.discourse-cdn.com/v4/letter/b/eb8c5e/32.png) [@billnbell](https://discuss.elastic.co/u/billnbell)\
**Post date:** [May 16, 2017, 2:55pm UTC](https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042/5 "2017-05-16T14:55:16Z")

</div>

I tried client.transport.sniff true and false and get the same issue. Node not available. Works fine in 5.3.2 TransportClient.

List nodes = client.connectedNodes();

This is empty....

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 13, 2017, 2:55pm UTC](https://discuss.elastic.co/t/elasticsearch-transport-client-fails-to-recovery-connection-after-cluster-restart/85042/6 "2017-06-13T14:55:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
