# Insert into elastic from spark: Connection error - all nodes failed

**URL:** <https://discuss.elastic.co/t/insert-into-elastic-from-spark-connection-error-all-nodes-failed/77657>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [March 7, 2017, 12:48pm UTC](https://discuss.elastic.co/t/insert-into-elastic-from-spark-connection-error-all-nodes-failed/77657 "2017-03-07T12:48:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Netanel\_Malka](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/netanel_malka/32/47520_2.png) [@Netanel\_Malka](https://discuss.elastic.co/u/Netanel_Malka)\
**Post date:** [March 7, 2017, 12:48pm UTC](https://discuss.elastic.co/t/insert-into-elastic-from-spark-connection-error-all-nodes-failed/77657/1 "2017-03-07T12:48:21Z")

</div>

Hi,  
I tried to insert 580 docs(~2MB size) to es but I got this message,  
when i tried to decrease the number of docs to 50 the job success.  
I am working with one spark partition, tried to tweak the batch size and the batch entries,  
but didn't succeed to overcome this...  
I also succeeded to connect to the es nodes from spark.

> org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed;[nodes with ip and port]

Some details:  
4 nodes of spark and elastic (4 cores 24GB, elastic heap size 2GB)  
spark 1.6.0  
es 2.3.3  
es-hadoop 2.4.2

Thanks.

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [March 12, 2017, 8:33pm UTC](https://discuss.elastic.co/t/insert-into-elastic-from-spark-connection-error-all-nodes-failed/77657/2 "2017-03-12T20:33:30Z")

</div>

@Netanel_Malka In 5.0 and above, the connector logs a sample of errors that it receives from the Elasticsearch cluster. In the mean time, I would increase logging levels to `TRACE` for your job and inspecting the error messages on the bulk responses. Also be sure to check your Elasticsearch node logs for rejections.

---

<div class="post-metadata">

**Author:** ![Netanel\_Malka](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/netanel_malka/32/47520_2.png) [@Netanel\_Malka](https://discuss.elastic.co/u/Netanel_Malka)\
**Post date:** [March 15, 2017, 9:43am UTC](https://discuss.elastic.co/t/insert-into-elastic-from-spark-connection-error-all-nodes-failed/77657/3 "2017-03-15T09:43:00Z")

</div>

Thanks for the respond.  
I found that **only** when I try to insert a doc with big(thousands of coordinates) geo\_shape(), I got this error.  
Also, i tried it in elasticsearch 5.2.2 with the suite connector, but the connector not write why it failed.  
I see that it finds the appropriate shards(DEBUG) and the partition writer instance on the right address, but then i got this exception again.  
Also, When i tried to delete the index i get acknowledge: false, although the index is deleted. But then i can't create any index until I restart the whole cluster...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2017, 9:43am UTC](https://discuss.elastic.co/t/insert-into-elastic-from-spark-connection-error-all-nodes-failed/77657/4 "2017-04-12T09:43:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
