# Elasticsearch 2.0 aws-cloud network issue **URL:** \ **Category:** Elasticsearch\ **Created:** [November 24, 2015, 6:24am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385 "2015-11-24T06:24:19Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Liran\_Shani](https://avatars.discourse-cdn.com/v4/letter/l/ba9def/32.png) [@Liran\_Shani](https://discuss.elastic.co/u/Liran_Shani)\ **Post date:** [November 24, 2015, 6:24am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/1 "2015-11-24T06:24:19Z")
I use aws (VPC) for ES cluster. After upgrading fron ES 1.7.2 to ES 2.0 i have network issues -\> [WARN][transport.netty] [es2-data.localdomain] exception caught on transport layer [[id: 0xd878a1ca, /10.111.112.87:38887 =\> /10.111.112.176:9300]], closing connection There is no nodes dicovery $: curl localhost:9200/\_nodes/process?pretty The problem is that the "transport\_address" : " **127.0.0.1:9300**" is set to localhost in **(ES 2.0)** where in **ES 1.7.2** the transport\_address is "transport\_address" : "**inet[/10.111.112.86:9300]**" My question is how to set correctly the network settings in ES 2.0 in order to use the private ip. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 24, 2015, 6:53am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/2 "2015-11-24T06:53:36Z")
Read this: [https://www.elastic.co/guide/en/elasticsearch/plugins/current/cloud-aws-discovery.html#cloud-aws-discovery-network-host](https://www.elastic.co/guide/en/elasticsearch/plugins/current/cloud-aws-discovery.html#cloud-aws-discovery-network-host) ---
**Author:** ![Liran\_Shani](https://avatars.discourse-cdn.com/v4/letter/l/ba9def/32.png) [@Liran\_Shani](https://discuss.elastic.co/u/Liran_Shani)\ **Post date:** [November 24, 2015, 7:08am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/3 "2015-11-24T07:08:48Z")
I read this, not sure how to use the proper settings. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 24, 2015, 7:23am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/4 "2015-11-24T07:23:22Z")
May be you missed the lines just above? ``` network.host: _ec2_ ``` Should work ---
**Author:** ![Liran\_Shani](https://avatars.discourse-cdn.com/v4/letter/l/ba9def/32.png) [@Liran\_Shani](https://discuss.elastic.co/u/Liran_Shani)\ **Post date:** [November 24, 2015, 7:44am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/5 "2015-11-24T07:44:11Z")
I tried this and got this exception: [2015-11-24 07:07:09,820][ERROR][bootstrap] Guice Exception: java.lang.IllegalArgumentException: No enum constant org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider.HostType._EC2_ My settings elasticsearch,yml ```auto cluster.name: ABC node.name: ABCDEFGH.localdomain node.master: true node.data: true node.max_local_storage_nodes: 1 index.mapper.dynamic: true action.auto_create_index: true action.disable_delete_all_indices: true bootstrap.mlockall: true http.port: 9200 network.host: _ec2 _ discovery.type: ec2 discovery.zen.ping.multicast.enabled: false cloud.aws.access_key: ABCDEFGH cloud.aws.secret_key: ABCDEFGH cloud.aws.region: eu-west-1 discovery.ec2.groups: ABCDEFGH discovery.ec2.availability_zones: ["eu-west-1a", "eu-west-1b"] ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 24, 2015, 8:00am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/6 "2015-11-24T08:00:41Z")
It sounds like a bug to me. Do you have a full stack trace? Could you try with `_ec2: privateIp_`? ---
**Author:** ![Liran\_Shani](https://avatars.discourse-cdn.com/v4/letter/l/ba9def/32.png) [@Liran\_Shani](https://discuss.elastic.co/u/Liran_Shani)\ **Post date:** [November 24, 2015, 8:13am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/7 "2015-11-24T08:13:18Z")
I tried with: network.host: \_ ec2:privateIp \_ could not do: ubuntu@es-data2:~$ curl localhost:9200/\_nodes/process?pretty curl: (7) Failed to connect to localhost port 9200: Connection refused stacktrace: [2015-11-24 08:03:18,819][INFO][node] [es2-data.localdomain] version[2.0.0], pid[6576], build[de54438/2015-10-22T08:09:48Z] [2015-11-24 08:03:18,820][INFO][node] [es2-data.localdomain] initializing ... [2015-11-24 08:03:19,087][INFO][plugins] [es2-data.localdomain] loaded [cloud-aws], sites [hq] [2015-11-24 08:03:19,122][INFO][env] [es2-data.localdomain] using [1] data paths, mounts [[/ (/dev/xvda1)]], net usable\_space [4.2gb], net total\_space [7.7gb], spins? [no], types [ext4] [2015-11-24 08:03:21,722][INFO][node] [es2-data.localdomain] initialized [2015-11-24 08:03:21,722][INFO][node] [es2-data.localdomain] starting ... [2015-11-24 08:03:21,821][INFO][transport] [es2-data.localdomain] publish\_address {10.111.112.87:9300}, bound\_addresses {10.111.112.87:9300} [2015-11-24 08:03:21,832][INFO][discovery] [es2-data.localdomain] ABCDEFGHtest/nIfqDAOYQ8WprHxFGeIxRQ [2015-11-24 08:03:22,665][WARN][transport.netty] [es2-data.localdomain] exception caught on transport layer [[id: 0x192d25cb, /10.111.112.87:58884 =\> /10.111.112.73:9300]], closing connection java.lang.NullPointerException at org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:206) at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:201) .. at java.lang.Thread.run(Thread.java:745) [2015-11-24 08:03:22,666][WARN][transport.netty] [es2-data.localdomain] exception caught on transport layer [[id: 0xe7228172, /10.111.112.87:37220 =\> /10.111.112.226:9300]], closing connection java.lang.NullPointerException .... [2015-11-24 08:03:22,665][WARN][discovery.zen.ping.unicast] [es2-data.localdomain] failed to send ping to [{#cloud-i-11e13da8-0}{10.111.112.121}{10.111.112.121:9300}] RemoteTransportException[[es1-data.localdomain][10.111.112.121:9300][internal:discovery/zen/unicast]]; nested: IllegalStateException[received ping request while not started]; Caused by: java.lang.IllegalStateException: received ping request while not started [2015-11-24 08:03:24,324][WARN][discovery.zen.ping.unicast] [es2-data.localdomain] failed to send ping to [{#cloud-i-11e13da8-0}{10.111.112.121}{10.111.112.121:9300}] RemoteTransportException[[es1-data.localdomain][10.111.112.121:9300][internal:discovery/zen/unicast]]; nested: IllegalStateException[received ping request while not started]; Caused by: java.lang.IllegalStateException: received ping request while not started at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.handlePingRequest(UnicastZenPing.java:478) at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.access$2400(UnicastZenPing.java:64) at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$UnicastPingRequestHandler.messageReceived(UnicastZenPing.java:503) ... [2015-11-24 08:03:27,517][INFO][cluster.service] [es2-data.localdomain] new\_master {es2-data.localdomain}{nIfqDAOYQ8WprHxFGeIxRQ}{10.111.112.87}{10.111.112.87:9300}{max\_local\_storage\_nodes=1, master=true}, reason: zen-disco-join(elected\_as\_master, [0] joins received) [2015-11-24 08:03:27,544][INFO][http] [es2-data.localdomain] publish\_address {10.111.112.87:9200}, bound\_addresses {10.111.112.87:9200} [2015-11-24 08:03:27,544][INFO][node] [es2-data.localdomain] started [2015-11-24 08:03:27,552][INFO][gateway] [es2-data.localdomain] recovered [0] indices into cluster\_state [2015-11-24 08:03:58,215][INFO][cluster.service] [es2-data.localdomain] added {{es1-data.localdomain}{K70lDLMYSe2Ok15X\_m9BUQ}{10.111.112.121}{10.111.112.121:9300}{max\_local\_storage\_nodes=1, master=true},}, reason: zen-disco-join(join from node[{es1-data.localdomain}{K70lDLMYSe2Ok15X\_m9BUQ}{10.111.112.121}{10.111.112.121:9300}{max\_local\_storage\_nodes=1, master=true}]) ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 24, 2015, 8:26am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/8 "2015-11-24T08:26:40Z")
Everything looks fine here. It bounds to `10.111.112.87` (ec2 private IP) so you need to run `curl 10.111.112.87:9200`. But could you run the same again with `_ec2_` and paste the stack trace? ---
**Author:** ![Liran\_Shani](https://avatars.discourse-cdn.com/v4/letter/l/ba9def/32.png) [@Liran\_Shani](https://discuss.elastic.co/u/Liran_Shani)\ **Post date:** [November 24, 2015, 10:33am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/9 "2015-11-24T10:33:19Z")
![](https://us1.discourse-cdn.com/elastic/original/2X/8/8c94e6caa9e193c9691c5d12f52c5111d2715818.png) with \_ ec2 \_ i had earlier a typeo, I am using now network.host: \_ ec2 \_ and see that the transport\_address is ok (the 2 nodes creates a cluster) but in the log there are still many exceptions: ```auto [2015-11-24 09:53:21,778][INFO][node] [es2-data.localdomain] version[2.0.0], pid[9509], build[de54438/2015-10-22T08:09:48Z] [2015-11-24 09:53:21,779][INFO][node] [es2-data.localdomain] initializing ... [2015-11-24 09:53:22,073][INFO][plugins] [es2-data.localdomain] loaded [cloud-aws], sites [hq] [2015-11-24 09:53:22,110][INFO][env] [es2-data.localdomain] using [1] data paths, mounts [[/ (/dev/xvda1)]], net usable_space [4.2gb], net total_space [7.7gb], spins? [no], types [ext4] [2015-11-24 09:53:24,687][INFO][node] [es2-data.localdomain] initialized [2015-11-24 09:53:24,687][INFO][node] [es2-data.localdomain] starting ... [2015-11-24 09:53:24,794][INFO][transport] [es2-data.localdomain] publish_address {10.111.112.87:9300}, bound_addresses {10.111.112.87:9300} [2015-11-24 09:53:24,803][INFO][discovery] [es2-data.localdomain] sisense-monitoring-cluster-test/9xDZmZXYRQOg2il-XOQ0KQ **[2015-11-24 09:53:25,693][WARN][transport.netty] [es2-data.localdomain] exception caught on transport layer [[id: 0x75753f9e, /10.111.112.87:59382 => /10.111.112.73:9300]], closing connection** **java.lang.NullPointerException** at org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:206) at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:201) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [2015-11-24 09:53:25,695][WARN][transport.netty] [es2-data.localdomain] exception caught on transport layer [[id: 0xc649b4e4, /10.111.112.87:55535 => /10.111.112.134:9300]], closing connection java.lang.NullPointerException at org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:206) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [2015-11-24 09:53:25,694][WARN][transport.netty] [es2-data.localdomain] exception caught on transport layer [[id: 0xfc3e4419, /10.111.112.87:37718 => /10.111.112.226:9300]], closing connection java.lang.NullPointerException at org.elasticsearch.transport.netty.MessageChannelHandler.handleException(MessageChannelHandler.java:206) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [2015-11-24 09:53:25,694][WARN][discovery.zen.ping.unicast] [es2-data.localdomain] failed to send ping to [{#cloud-i-11e13da8-0}{10.111.112.121}{10.111.112.121:9300}] RemoteTransportException[[es1-data.localdomain][10.111.112.121:9300][internal:discovery/zen/unicast]]; nested: IllegalStateException[received ping request while not started]; Caused by: java.lang.IllegalStateException: received ping request while not started at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.handlePingRequest(UnicastZenPing.java:478) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [2015-11-24 09:53:25,698][WARN][transport.netty] [es2-data.localdomain] exception caught on transport layer [[id: 0x91df9511, /10.111.112.87:46313 => /10.111.112.86:9300]], closing connection java.lang.NullPointerException at java.lang.Thread.run(Thread.java:745) [2015-11-24 09:53:30,632][INFO][cluster.service] [es2-data.localdomain] new_master {es2-data.localdomain}{9xDZmZXYRQOg2il-XOQ0KQ}{10.111.112.87}{10.111.112.87:9300}{max_local_storage_nodes=1}, reason: zen-disco-join(elected_as_master, [0] joins received) [2015-11-24 09:53:30,660][INFO][http] [es2-data.localdomain] publish_address {10.111.112.87:9200}, bound_addresses {10.111.112.87:9200} [2015-11-24 09:53:30,660][INFO][node] [es2-data.localdomain] started [2015-11-24 09:53:30,665][INFO][gateway] [es2-data.localdomain] recovered [0] indices into cluster_state [2015-11-24 09:54:01,004][INFO][cluster.service] [es2-data.localdomain] added {{es1-data.localdomain}{TfgeBr1fR3StfHQdpXiPbQ}{10.111.112.121}{10.111.112.121:9300}{max_local_storage_nodes=1},}, reason: zen-disco-join(join from node[{es1-data.localdomain}{TfgeBr1fR3StfHQdpXiPbQ}{10.111.112.121}{10.111.112.121:9300}{max_local_storage_nodes=1}]) ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 24, 2015, 4:04pm UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/10 "2015-11-24T16:04:04Z")
You are mixing node versions apparently. Some "older" nodes are trying to communicate with this new one. See [https://github.com/elastic/elasticsearch/issues/14400](https://github.com/elastic/elasticsearch/issues/14400) ---
**Author:** ![Liran\_Shani](https://avatars.discourse-cdn.com/v4/letter/l/ba9def/32.png) [@Liran\_Shani](https://discuss.elastic.co/u/Liran_Shani)\ **Post date:** [November 26, 2015, 6:36am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/11 "2015-11-26T06:36:32Z")
I will check it. Indeed i am trying to upgrade from 1.7 to 2.0. There might be running ES v1.7 Will update... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:35pm UTC](https://discuss.elastic.co/t/elasticsearch-2-0-aws-cloud-network-issue/35385/12 "2017-07-05T23:35:44Z")