# Handshake failure (TransportClientNodesService: failed to connect to node)

**URL:** <https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787>\
**Category:** Elasticsearch\
**Created:** [April 18, 2017, 10:10pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787 "2017-04-18T22:10:45Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![esearch](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@esearch](https://discuss.elastic.co/u/esearch)\
**Post date:** [April 18, 2017, 10:10pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/1 "2017-04-18T22:10:45Z")

</div>

I'm a little stuck figuring out why I'm getting a handshake failure when trying to connect to ES. I ran ES with the following command:

`docker run -e ES_JAVA_OPTS='-Xms1g -Xmx1g' --name elasticsearch -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" -e "xpack.security.enabled=false" -d -p 9200:9200 -p 9300:9300 docker.elastic.co/elasticsearch/elasticsearch:5.2.2`

When I access ES from the browser at localhost:9200, I get:

```auto
"name": "zW-D1Xo",
"cluster_name": "docker-cluster",
"cluster_uuid": "j9DGTipFSB6qC5P-LWvxPQ",
"version": {
"number": "5.2.2",
"build_hash": "f9d9b74",
"build_date": "2017-02-24T17:26:45.835Z",
"build_snapshot": false,
"lucene_version": "6.4.1"
},
"tagline": "You Know, for Search"
}

```

I set up the TransportClient like so:

```auto
transportClient = new PreBuiltTransportClient(settingsBuilder.build())
          .addTransportAddresses(transportAddresses);

```

The only setting is [cluster.name](http://cluster.name).

The error I get is this:

```auto
INFO [2017-04-18 21:38:31,593] org.elasticsearch.client.transport.TransportClientNodesService: failed to connect to node [{#transport#-1}{oFK78nJYT9-FGrQbHEk-qg}{localhost}{127.0.0.1:9300}], removed from nodes list
! org.elasticsearch.transport.TransportException: connection reset
! at org.elasticsearch.transport.TcpTransport.onChannelClosed(TcpTransport.java:1610)
! at org.elasticsearch.transport.netty4.Netty4Transport.access$100(Netty4Transport.java:93)
! at org.elasticsearch.transport.netty4.Netty4Transport$ChannelCloseListener.operationComplete(Netty4Transport.java:412)
! at org.elasticsearch.transport.netty4.Netty4Transport$ChannelCloseListener.operationComplete(Netty4Transport.java:402)
! at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
! at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
! at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
! at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
! at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
! at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1058)
! at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:686)
! at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:664)
! at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:607)
! at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:82)
! at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:137)
! at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)
! at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:527)
! at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:481)
! at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
! at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
! ... 1 common frames omitted
! Causing: java.lang.IllegalStateException: handshake failed
! at org.elasticsearch.transport.TcpTransport.executeHandshake(TcpTransport.java:1568)
! at org.elasticsearch.transport.TcpTransport.openConnection(TcpTransport.java:502)
! ... 9 common frames omitted
! Causing: org.elasticsearch.transport.ConnectTransportException: [][127.0.0.1:9300] general node connection failure
! at org.elasticsearch.transport.TcpTransport.openConnection(TcpTransport.java:519)
! at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:460)
! at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:318)
! at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.doSample(TransportClientNodesService.java:408)
! at org.elasticsearch.client.transport.TransportClientNodesService$NodeSampler.sample(TransportClientNodesService.java:354)
! at org.elasticsearch.client.transport.TransportClientNodesService$ScheduledNodeSampler.run(TransportClientNodesService.java:387)
! at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:527)
! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
! at java.lang.Thread.run(Thread.java:745)

```

Here is the Maven dependency (where dep.es.version is 5.2.2):

```auto
    <dependency>
      <groupId>org.elasticsearch.client</groupId>
      <artifactId>transport</artifactId>
      <version>${dep.es.version}</version>
      <exclusions>
        <exclusion>
          <groupId>joda-time</groupId>
          <artifactId>joda-time</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.apache.httpcomponents</groupId>
          <artifactId>httpcore</artifactId>
        </exclusion>
        <exclusion>
          <groupId>commons-logging</groupId>
          <artifactId>commons-logging</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.elasticsearch</groupId>
      <artifactId>elasticsearch</artifactId>
      <version>${dep.es.version}</version>
      <exclusions>
        <exclusion>
          <groupId>joda-time</groupId>
          <artifactId>joda-time</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.apache.httpcomponents</groupId>
          <artifactId>httpcore</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

```

I don't know what I should try next. Any ideas or directions I should pursue?

Thanks,  
Benson

---

<div class="post-metadata">

**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\
**Post date:** [April 21, 2017, 8:08am UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/2 "2017-04-21T08:08:40Z")

</div>

> [@esearch](#):
>
> transportClient = new PreBuiltTransportClient(settingsBuilder.build())  
> .addTransportAddresses(transportAddresses);

are you sure that `transportAddresses` are correct? and do you see an logs on the server?

> [@esearch](#):
>
> -p 9200:9200 -p 9300:9300

I am just guessing since I am not familiar with this but specifying the same parameter twice looks suspicious?

---

<div class="post-metadata">

**Author:** ![esearch](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@esearch](https://discuss.elastic.co/u/esearch)\
**Post date:** [April 21, 2017, 4:38pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/3 "2017-04-21T16:38:15Z")

</div>

Thanks for responding.

> [@s1monw](#):
>
> > [@esearch](#):
> >
> > transportClient = new PreBuiltTransportClient(settingsBuilder.build())  
> > .addTransportAddresses(transportAddresses);
> 
> are you sure that `transportAddresses` are correct? and do you see an logs on the server?

The transportAddresses are just localhost (I added a printout in my code below):

```
ADDRESS: [127.0.0.1:9300]
INFO [2017-04-21 16:20:30,478] org.elasticsearch.client.transport.TransportClientNodesService: failed to connect to node [{#transport#-1}{8qMGuDVmR-WP94YLe4WqYg}{localhost}{127.0.0.1:9300}], removed from nodes list

```

As for the server, I'm not seeing anything on the elasticsearch side. Here are the total logs (this time I didn't load Kibana) in the docker container:

```auto
[2017-04-21T16:19:23,657][INFO][o.e.n.Node] [] initializing ...
[2017-04-21T16:19:23,766][INFO][o.e.e.NodeEnvironment] [4NynSsv] using [1] data paths, mounts [[/ (overlay)]], net usable_space [57.4gb], net total_space [62.7gb], spins? [possibly], types [overlay]
[2017-04-21T16:19:23,766][INFO][o.e.e.NodeEnvironment] [4NynSsv] heap size [990.7mb], compressed ordinary object pointers [true]
[2017-04-21T16:19:23,767][INFO][o.e.n.Node] node name [4NynSsv] derived from node ID [4NynSsv9RHS8EYIvevF1vQ]; set [node.name] to override
[2017-04-21T16:19:23,771][INFO][o.e.n.Node] version[5.2.2], pid[1], build[f9d9b74/2017-02-24T17:26:45.835Z], OS[Linux/4.9.21-moby/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_92-internal/25.92-b14]
[2017-04-21T16:19:25,370][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [aggs-matrix-stats]
[2017-04-21T16:19:25,370][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [ingest-common]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [lang-expression]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [lang-groovy]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [lang-mustache]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [lang-painless]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [percolator]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [reindex]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [transport-netty3]
[2017-04-21T16:19:25,371][INFO][o.e.p.PluginsService] [4NynSsv] loaded module [transport-netty4]
[2017-04-21T16:19:25,372][INFO][o.e.p.PluginsService] [4NynSsv] loaded plugin [x-pack]
[2017-04-21T16:19:25,783][WARN][o.e.d.s.g.GroovyScriptEngineService] [groovy] scripts are deprecated, use [painless] scripts instead
[2017-04-21T16:19:28,381][INFO][o.e.n.Node] initialized
[2017-04-21T16:19:28,381][INFO][o.e.n.Node] [4NynSsv] starting ...
[2017-04-21T16:19:28,653][WARN][i.n.u.i.MacAddressUtil] Failed to find a usable hardware address from the network interfaces; using random bytes: 59:29:07:60:83:8c:94:96
[2017-04-21T16:19:28,730][INFO][o.e.t.TransportService] [4NynSsv] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2017-04-21T16:19:31,826][INFO][o.e.c.s.ClusterService] [4NynSsv] new_master {4NynSsv}{4NynSsv9RHS8EYIvevF1vQ}{_A8_cvJKQDirLQTHdntn7Q}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-04-21T16:19:31,857][INFO][o.e.h.HttpServer] [4NynSsv] publish_address {172.17.0.2:9200}, bound_addresses {0.0.0.0:9200}
[2017-04-21T16:19:31,857][INFO][o.e.n.Node] [4NynSsv] started
[2017-04-21T16:19:31,939][INFO][o.e.g.GatewayService] [4NynSsv] recovered [0] indices into cluster_state
[2017-04-21T16:19:32,369][INFO][o.e.l.LicenseService] [4NynSsv] license [e10d1dc3-f224-4806-bcc2-7dbef9219ddb] mode [trial] - valid
[2017-04-21T16:19:38,619][INFO][o.e.c.m.MetaDataCreateIndexService] [4NynSsv] [.monitoring-es-2-2017.04.21] creating index, cause [auto(bulk api)], templates [.monitoring-es-2], shards [1]/[1], mappings [shards, _default_, node, index_stats, index_recovery, cluster_state, cluster_stats, indices_stats, node_stats]
[2017-04-21T16:19:38,733][INFO][o.e.c.m.MetaDataCreateIndexService] [4NynSsv] [.monitoring-data-2] creating index, cause [auto(bulk api)], templates [.monitoring-data-2], shards [1]/[1], mappings [logstash, node, _default_, kibana, cluster_info]
[2017-04-21T16:19:38,983][INFO][o.e.c.m.MetaDataMappingService] [4NynSsv] [.monitoring-es-2-2017.04.21/b2GEwCVlSiidTF_QcpAs2A] update_mapping [cluster_stats]
[2017-04-21T16:19:39,037][INFO][o.e.c.m.MetaDataMappingService] [4NynSsv] [.monitoring-es-2-2017.04.21/b2GEwCVlSiidTF_QcpAs2A] update_mapping [node_stats]
[2017-04-21T16:19:49,185][INFO][o.e.c.m.MetaDataMappingService] [4NynSsv] [.monitoring-es-2-2017.04.21/b2GEwCVlSiidTF_QcpAs2A] update_mapping [index_stats]
[2017-04-21T16:19:49,266][INFO][o.e.c.m.MetaDataMappingService] [4NynSsv] [.monitoring-es-2-2017.04.21/b2GEwCVlSiidTF_QcpAs2A] update_mapping [cluster_stats]
[2017-04-21T16:19:49,319][INFO][o.e.c.m.MetaDataMappingService] [4NynSsv] [.monitoring-es-2-2017.04.21/b2GEwCVlSiidTF_QcpAs2A] update_mapping [indices_stats]

```

> [@s1monw](#):
>
> > [@esearch](#):
> >
> > -p 9200:9200 -p 9300:9300
> 
> I am just guessing since I am not familiar with this but specifying the same parameter twice looks suspicious?

You can expose multiple ports:

> <https://stackoverflow.com/questions/20845056/how-can-i-expose-more-than-1-port-with-docker>

Any other thoughts?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![esearch](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@esearch](https://discuss.elastic.co/u/esearch)\
**Post date:** [April 21, 2017, 4:42pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/4 "2017-04-21T16:42:55Z")

</div>

I'm not sure if this is useful, but I tried curling the ports directly:

```auto
$ curl localhost:9200
{
  "name" : "4NynSsv",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "kOhN_OIrRoyAUc15sJEs7A",
  "version" : {
    "number" : "5.2.2",
    "build_hash" : "f9d9b74",
    "build_date" : "2017-02-24T17:26:45.835Z",
    "build_snapshot" : false,
    "lucene_version" : "6.4.1"
  },
  "tagline" : "You Know, for Search"
}
$ curl localhost:9300
curl: (52) Empty reply from server
$ curl localhost:9400
curl: (7) Failed to connect to localhost port 9400: Connection refused

```

I did the last one just to see if I would get a different reply for a invalid port.

---

<div class="post-metadata">

**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\
**Post date:** [April 21, 2017, 6:26pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/5 "2017-04-21T18:26:59Z")

</div>

> [@esearch](#):
>
> curl localhost:9300  
> curl: (52) Empty reply from server

hmm we usually try to detect if you are trying to talk to a tranport port via HTTP so you should get a reply with a line like this: `This is not a HTTP port` do you get this if you do `curl -XGET http://localhost:9300`?

---

<div class="post-metadata">

**Author:** ![esearch](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@esearch](https://discuss.elastic.co/u/esearch)\
**Post date:** [April 21, 2017, 6:52pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/6 "2017-04-21T18:52:16Z")

</div>

I get the same error as before:

```auto
$ curl -XGET http://localhost:9300
curl: (52) Empty reply from server

```

---

<div class="post-metadata">

**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\
**Post date:** [April 21, 2017, 6:55pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/7 "2017-04-21T18:55:35Z")

</div>

when I do this I get this on 5.3

```auto
curl -XGET http://localhost:9300
This is not a HTTP port

```

---

<div class="post-metadata">

**Author:** ![esearch](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@esearch](https://discuss.elastic.co/u/esearch)\
**Post date:** [April 21, 2017, 7:03pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/8 "2017-04-21T19:03:25Z")

</div>

So all I ran was the docker command:

```auto
docker run -e ES_JAVA_OPTS='-Xms1g -Xmx1g' --name elasticsearch -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" -e "xpack.security.enabled=false" -d -p 9200:9200 -p 9300:9300 docker.elastic.co/elasticsearch/elasticsearch:5.2.2

```

and that curl. I'm on a Mac. Not sure if that makes a difference. I'm also using docker beta:

```auto
Version 17.05.0-ce-rc1-mac8 (16582)
Channel: edge
73d01bb48e

```

What are your specs?

---

<div class="post-metadata">

**Author:** ![esearch](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@esearch](https://discuss.elastic.co/u/esearch)\
**Post date:** [April 21, 2017, 7:10pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/9 "2017-04-21T19:10:05Z")

</div>

OK, if I do the curl INSIDE the docker container, I get the same error that you get.

---

<div class="post-metadata">

**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\
**Post date:** [April 21, 2017, 7:56pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/10 "2017-04-21T19:56:29Z")

</div>

> [@esearch](#):
>
> -e "transport.host=127.0.0.1"

what if you set this to `0.0.0.0` as well?

---

<div class="post-metadata">

**Author:** ![esearch](https://avatars.discourse-cdn.com/v4/letter/e/5fc32e/32.png) [@esearch](https://discuss.elastic.co/u/esearch)\
**Post date:** [April 21, 2017, 8:58pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/11 "2017-04-21T20:58:53Z")

</div>

That fixed everything. Thanks!!!!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 19, 2017, 9:03pm UTC](https://discuss.elastic.co/t/handshake-failure-transportclientnodesservice-failed-to-connect-to-node/82787/12 "2017-05-19T21:03:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
