# Getting a "No data nodes with HTTP-enabled available" error when writing from Spark to elasticsearch on Google Dataproc **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [August 10, 2017, 2:53pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631 "2017-08-10T14:53:11Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![bw2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bw2/32/25266_2.png) [@bw2](https://discuss.elastic.co/u/bw2)\ **Post date:** [August 10, 2017, 2:53pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/1 "2017-08-10T14:53:11Z")
I'm trying to export data from Spark =\> elasticsearch cluster running on Google Container Engine (GKE). I've deployed an ES cluster using configs from [https://github.com/pires/kubernetes-elasticsearch-cluster/tree/master/stateful](https://github.com/pires/kubernetes-elasticsearch-cluster/tree/master/stateful) that create a couple of each node type: master, client, data. I'm able to insert data into ES through the Spark connector if I have it connect to one of the Client nodes and set es.nodes.client.only=true After reading [https://www.elastic.co/guide/en/elasticsearch/hadoop/master/configuration.html#\_network](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/configuration.html#_network) and [https://www.elastic.co/guide/en/elasticsearch/hadoop/master/cloud.html](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/cloud.html) though, I'd like to have Spark write directly to the data nodes. However, if I switch back to the default es.nodes.client.only=false I get this error: ```auto at java.lang.Thread.run(Thread.java:748)org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No data nodes with HTTP-enabled available at org.elasticsearch.hadoop.rest.InitializationUtils.filterNonDataNodesIfNeeded(InitializationUtils.java:157) at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:576) at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:58) at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:91) at org.elasticsearch.spark.sql.EsSparkSQL$$anonfun$saveToEs$1.apply(EsSparkSQL.scala:91) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70) at org.apache.spark.scheduler.Task.run(Task.scala:86) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Error summary: EsHadoopIllegalArgumentException: No data nodes with HTTP-enabled available ``` ---
**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\ **Post date:** [August 11, 2017, 5:32pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/2 "2017-08-11T17:32:23Z")
I'm not too well versed in Docker, but taking a look through the configurations for the linked deployment I found these lines which are probably responsible for the lack of HTTP availability: > I also only see network configurations for transport level traffic (port 9300, non-http). I don't think this configuration is meant to be used in the manner that you are describing. ---
**Author:** ![bw2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bw2/32/25266_2.png) [@bw2](https://discuss.elastic.co/u/bw2)\ **Post date:** [August 11, 2017, 8:55pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/3 "2017-08-11T20:55:29Z")
> [@james.baiera](#): > > name: HTTP\_ENABLE > value: "false" Thanks, toggling that to "true" does fix it. ---
**Author:** ![bw2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bw2/32/25266_2.png) [@bw2](https://discuss.elastic.co/u/bw2)\ **Post date:** [August 11, 2017, 8:57pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/4 "2017-08-11T20:57:18Z")
I'm not sure how much of a difference this makes, but isn't it suboptimal that the Elasticsearch Spark connector communicates with elasticsearch using HTTP rather than the TCP protocol on 9300? ---
**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\ **Post date:** [August 15, 2017, 5:14pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/5 "2017-08-15T17:14:29Z")
TCP protocol is not backwards compatible between versions of Elasticsearch the same way that HTTP is. There have also been a fair amount of benchmarks performed on HTTP vs RPC and they have found that the two have comparable performance characteristics. ---
**Author:** ![bw2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bw2/32/25266_2.png) [@bw2](https://discuss.elastic.co/u/bw2)\ **Post date:** [August 16, 2017, 2:03am UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/6 "2017-08-16T02:03:04Z")
Got it. Thanks. ---
**Author:** ![tempogrup](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tempogrup/32/21913_2.png) [@tempogrup](https://discuss.elastic.co/u/tempogrup)\ **Post date:** [September 8, 2017, 12:30pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/7 "2017-09-08T12:30:13Z")
Thank you for sharing! [ankara temizlik şirketleri](http://tempogrup.net/) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 6, 2017, 12:30pm UTC](https://discuss.elastic.co/t/getting-a-no-data-nodes-with-http-enabled-available-error-when-writing-from-spark-to-elasticsearch-on-google-dataproc/96631/8 "2017-10-06T12:30:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.