# Get org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried

**URL:** <https://discuss.elastic.co/t/get-org-elasticsearch-hadoop-rest-eshadoopnonodesleftexception-connection-error-check-network-and-or-proxy-settings-all-nodes-failed-tried/79509>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [March 21, 2017, 9:39pm UTC](https://discuss.elastic.co/t/get-org-elasticsearch-hadoop-rest-eshadoopnonodesleftexception-connection-error-check-network-and-or-proxy-settings-all-nodes-failed-tried/79509 "2017-03-21T21:39:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Min\_Qiu](https://avatars.discourse-cdn.com/v4/letter/m/e79b87/32.png) [@Min\_Qiu](https://discuss.elastic.co/u/Min_Qiu)\
**Post date:** [March 21, 2017, 9:39pm UTC](https://discuss.elastic.co/t/get-org-elasticsearch-hadoop-rest-eshadoopnonodesleftexception-connection-error-check-network-and-or-proxy-settings-all-nodes-failed-tried/79509/1 "2017-03-21T21:39:24Z")

</div>

got org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; error

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 2, localhost, executor driver): org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[[elastic1-sea.cdx-dev.unifieddeliverynetwork.net:9200](http://elastic1-sea.cdx-dev.unifieddeliverynetwork.net:9200)]]  
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:123)  
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:303)  
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:287)  
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:291)  
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:118)  
at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:367)  
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:84)  
......  
Driver stacktrace:  
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)  
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)  
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)  
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)  
at scala.Option.foreach(Option.scala:245)  
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)  
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)  
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)  
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)  
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)  
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)  
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1918)  
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1931)  
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1951)  
at org.elasticsearch.spark.rdd.EsSpark$.saveToEs(EsSpark.scala:34)  
at org.elasticsearch.spark.rdd.EsSpark$.saveToEs(EsSpark.scala:27)  
at org.elasticsearch.spark.rdd.EsSpark$.saveJsonToEs(EsSpark.scala:40)  
at org.elasticsearch.spark.rdd.api.java.JavaEsSpark$.saveJsonToEs(JavaEsSpark.scala:32)  
at org.elasticsearch.spark.rdd.api.java.JavaEsSpark.saveJsonToEs(JavaEsSpark.scala)  
at com.vidscal.analysics.readmessage.readMessageFromKafkaToES$2.call(readMessageFromKafkaToES.java:182)  
at com.vidscal.analysics.readmessage.readMessageFromKafkaToES$2.call(readMessageFromKafkaToES.java:165)  
at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272)  
at org.apache.spark.streaming.api.java.JavaDStreamLike$$anonfun$foreachRDD$1.apply(JavaDStreamLike.scala:272)  
at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:627)  
at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:627)  
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51)  
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)  
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51)  
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)  
at .....  
org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254)  
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)  
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254)  
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)  
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  
at java.lang.Thread.run(Thread.java:745)  
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[[elastic1-sea.cdx-dev.unifieddeliverynetwork.net:9200](http://elastic1-sea.cdx-dev.unifieddeliverynetwork.net:9200)]]  
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:123)  
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:303)  
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:287)  
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:291)  
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:118)  
at org.elasticsearch.hadoop.rest.RestClient.esVersion(RestClient.java:367)  
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:84)  
at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:347)  
at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:31)  
at org.elasticsearch.spark.rdd.EsSpark$$anonfun$saveToEs$1.apply(EsSpark.scala:34)  
at org.elasticsearch.spark.rdd.EsSpark$$anonfun$saveToEs$1.apply(EsSpark.scala:34)  
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)  
at org.apache.spark.scheduler.Task.run(Task.scala:99)  
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)  
... 3 more

my configuration is

```
    sparkConf.set("es.index.auto.create", "true");
    sparkConf.set("es.nodes","XXXXXXXX");
    sparkConf.set("es.port","9200");
    sparkConf.set("es.nodes.client.only","true");
    sparkConf.set("es.nodes.discovery", "false");
    sparkConf.set("es.net.ssl","true");
    sparkConf.set("es.net.http.auth.user","XXXX");
    sparkConf.set("es.net.http.auth.pass","XXXXX");

   .......
    JavaEsSpark.saveJsonToEs(rdd, "min_test/test1");
   .......

```

In the spark cluster, I am able to do curl -XGET "XXXXXX:9200/min\_test/test1"

But unable to do ping XXXXXX, The Elastic Cluster is in AWS.

What should I do? Please advise.

Thanks

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [March 22, 2017, 1:56pm UTC](https://discuss.elastic.co/t/get-org-elasticsearch-hadoop-rest-eshadoopnonodesleftexception-connection-error-check-network-and-or-proxy-settings-all-nodes-failed-tried/79509/2 "2017-03-22T13:56:42Z")

</div>

If your Elasticsearch cluster is hosted in AWS, my guess would be that you need to tune your firewall settings, or set `es.nodes.wan.only` to `true` and remove the `es.nodes.client.only` and `es.nodes.discovery` settings.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 19, 2017, 1:56pm UTC](https://discuss.elastic.co/t/get-org-elasticsearch-hadoop-rest-eshadoopnonodesleftexception-connection-error-check-network-and-or-proxy-settings-all-nodes-failed-tried/79509/3 "2017-04-19T13:56:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
