# Writing from spark to elasticsearch fails

**URL:** <https://discuss.elastic.co/t/writing-from-spark-to-elasticsearch-fails/92172>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [July 6, 2017, 9:25pm UTC](https://discuss.elastic.co/t/writing-from-spark-to-elasticsearch-fails/92172 "2017-07-06T21:25:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jobyjohny](https://avatars.discourse-cdn.com/v4/letter/j/eada6e/32.png) [@jobyjohny](https://discuss.elastic.co/u/jobyjohny)\
**Post date:** [July 6, 2017, 9:25pm UTC](https://discuss.elastic.co/t/writing-from-spark-to-elasticsearch-fails/92172/1 "2017-07-06T21:25:48Z")

</div>

I am using a single node cluster ,installed elastic search and kibana on my local machine.  
Created the logic in spark to read the csv files and create dataframes.While writing those into elasticsearch ,it is throwing an error

**Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'**  
\*\* at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:250)\*\*

conf settings given in the code are given below  
spark.conf.set("es.index.auto.create", "true")  
spark.conf.set("es.nodes.wan.only", "true")  
spark.conf.set("es.nodes", "127.0.0.1")  
spark.conf.set("es.port", "9200")

val cases\_count\_stats = spark.sql("select ...from cases\_txn\_cust\_table group by case\_identifier,case\_description")  
println("Writing to elastic")

cases\_count\_stats.saveToEs("case\_summary1/cs\_type")

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [July 17, 2017, 2:14am UTC](https://discuss.elastic.co/t/writing-from-spark-to-elasticsearch-fails/92172/2 "2017-07-17T02:14:02Z")

</div>

This exception almost always has to do with your Spark driver or the executors not being able to reach your Elasticsearch cluster. If you are running with wan-only mode enabled against localhost, make sure that spark is only executing on your local machine. Also make sure that Elasticsearch is up and running and responding to simple requests. Attaching the `TRACE` level logs also might help in troubleshooting these problems.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 14, 2017, 2:14am UTC](https://discuss.elastic.co/t/writing-from-spark-to-elasticsearch-fails/92172/3 "2017-08-14T02:14:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.