# Unable to write data to elasticsearch 5.3 from Databricks/Spark

**URL:** <https://discuss.elastic.co/t/unable-to-write-data-to-elasticsearch-5-3-from-databricks-spark/85299>\
**Category:** Elasticsearch\
**Created:** [May 10, 2017, 7:57pm UTC](https://discuss.elastic.co/t/unable-to-write-data-to-elasticsearch-5-3-from-databricks-spark/85299 "2017-05-10T19:57:12Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bobby259](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@bobby259](https://discuss.elastic.co/u/bobby259)\
**Post date:** [May 10, 2017, 7:57pm UTC](https://discuss.elastic.co/t/unable-to-write-data-to-elasticsearch-5-3-from-databricks-spark/85299/1 "2017-05-10T19:57:12Z")

</div>

I am facing issue while trying to write data from Databricks Platform to Elasticsearch 5.3 cluster. Below is the error, please help.

```
    17/05/10 19:50:13 ERROR Executor: Exception in task 14.1 in stage 533.0 (TID 37464)
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
	at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:247)
	at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:545)
	at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:58)
	at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:94)
	at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:94)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[54.234.233.217:9200]] 
	at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:150)
	at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:444)
	at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:424)
	at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:428)
	at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:154)
	at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:609)
	at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:240)
```

---

<div class="post-metadata">

**Author:** ![bobby259](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@bobby259](https://discuss.elastic.co/u/bobby259)\
**Post date:** [May 10, 2017, 8:51pm UTC](https://discuss.elastic.co/t/unable-to-write-data-to-elasticsearch-5-3-from-databricks-spark/85299/2 "2017-05-10T20:51:45Z")

</div>

I have made some changes to my elasticsearch.yml file and now I see this:

```
java.net.PlainSocketImpl.socketConnect(Native Method)
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
java.net.Socket.connect(Socket.java:589)
java.net.Socket.connect(Socket.java:538)
java.net.Socket.<init>(Socket.java:434)
java.net.Socket.<init>(Socket.java:286)
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:471)
org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:112)
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:444)
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:424)
org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:428)
org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:154)
org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:609)
org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:240)
org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:545)
org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:58)
org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:94)
org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:94)
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
org.apache.spark.scheduler.Task.run(Task.scala:99)
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
```

---

<div class="post-metadata">

**Author:** ![bobby259](https://avatars.discourse-cdn.com/v4/letter/b/9de053/32.png) [@bobby259](https://discuss.elastic.co/u/bobby259)\
**Post date:** [May 10, 2017, 9:39pm UTC](https://discuss.elastic.co/t/unable-to-write-data-to-elasticsearch-5-3-from-databricks-spark/85299/3 "2017-05-10T21:39:45Z")

</div>

Guys, did anyone face this issue? What could be the possible solution? Any help is appreciated:

```
17/05/10 21:31:29 ERROR Executor: Exception in task 9.0 in stage 60.0 (TID 4095)
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
	at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:247)
	at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:545)
	at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:58)
	at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:94)
	at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:94)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[54.234.233.217:9200]] 
	at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:150)
	at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:444)
	at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:424)
	at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:428)
	at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:154)
	at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:609)
	at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:240)
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 7, 2017, 9:53pm UTC](https://discuss.elastic.co/t/unable-to-write-data-to-elasticsearch-5-3-from-databricks-spark/85299/4 "2017-06-07T21:53:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.