# How to config elasticsearch nodes remote in pyspark? Node \[127.0.0.1:9200\] failed (Connection refused (Connection refused)); no other nodes left - aborting

**URL:** <https://discuss.elastic.co/t/how-to-config-elasticsearch-nodes-remote-in-pyspark-node-127-0-0-1-9200-failed-connection-refused-connection-refused-no-other-nodes-left-aborting/131735>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [May 14, 2018, 12:11pm UTC](https://discuss.elastic.co/t/how-to-config-elasticsearch-nodes-remote-in-pyspark-node-127-0-0-1-9200-failed-connection-refused-connection-refused-no-other-nodes-left-aborting/131735 "2018-05-14T12:11:21Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Luan\_Ha\_Thanh](https://avatars.discourse-cdn.com/v4/letter/l/848f3c/32.png) [@Luan\_Ha\_Thanh](https://discuss.elastic.co/u/Luan_Ha_Thanh)\
**Post date:** [May 14, 2018, 12:11pm UTC](https://discuss.elastic.co/t/how-to-config-elasticsearch-nodes-remote-in-pyspark-node-127-0-0-1-9200-failed-connection-refused-connection-refused-no-other-nodes-left-aborting/131735/1 "2018-05-14T12:11:21Z")

</div>

I want to connect Spark to Elasticsearch. I use command:

> pyspark --driver-class-path /home/bigdata/elasticsearch-hadoop-5.6.5/elasticsearch-hadoop-5.6.5/dist/elasticsearch-spark-20\_2.10-5.6.5.jar --conf spark.es.nodes=107.111.111.111 --conf spark.es.port=9200

But it don't connect:

> Py4JJavaError: An error occurred while calling o43.save.  
> : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 5.0 failed 1 times, most recent failure: Lost task 0.0 in stage 5.0 (TID 5, localhost, executor driver): org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[127.0.0.1:9200]]  
> ERROR NetworkClient

> Node [127.0.0.1:9200] failed (Connection refused (Connection refused)); no other nodes left - aborting...

**(But I want to connect to 107.111.111.111:9200, not to localhost:9200).**

**My code (in Python Jupyter Notebook):**

> PATH\_TO\_DATA = "../elasticsearch-spark-recommender/data/ml-latest-small"  
> ratings = spark.read.csv(PATH\_TO\_DATA + "/ratings.csv", header=True, inferSchema=True)  
> ratings.cache()  
> print("Number of ratings: %i" % ratings.count())  
> print("Sample of ratings:")  
> ratings.show(5)
> 
> ratings.write.format("es").save("demo/ratings")

Can you help me?  
Thank you very much.

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [May 30, 2018, 7:35pm UTC](https://discuss.elastic.co/t/how-to-config-elasticsearch-nodes-remote-in-pyspark-node-127-0-0-1-9200-failed-connection-refused-connection-refused-no-other-nodes-left-aborting/131735/2 "2018-05-30T19:35:12Z")

</div>

I am not too familiar with how Pyspark captures settings as opposed to how the standard Spark connector captures them. Have you tried the settings without the `spark.` prefix?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 27, 2018, 7:35pm UTC](https://discuss.elastic.co/t/how-to-config-elasticsearch-nodes-remote-in-pyspark-node-127-0-0-1-9200-failed-connection-refused-connection-refused-no-other-nodes-left-aborting/131735/3 "2018-06-27T19:35:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.