# ES Spark API writing to a Cluster failes when connecting to CLient Node

**URL:** <https://discuss.elastic.co/t/es-spark-api-writing-to-a-cluster-failes-when-connecting-to-client-node/34697>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [November 16, 2015, 4:04pm UTC](https://discuss.elastic.co/t/es-spark-api-writing-to-a-cluster-failes-when-connecting-to-client-node/34697 "2015-11-16T16:04:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ramdev\_Wudali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramdev_wudali/32/7894_2.png) [@Ramdev\_Wudali](https://discuss.elastic.co/u/Ramdev_Wudali)\
**Post date:** [November 16, 2015, 4:04pm UTC](https://discuss.elastic.co/t/es-spark-api-writing-to-a-cluster-failes-when-connecting-to-client-node/34697/1 "2015-11-16T16:04:34Z")

</div>

Hi:  
I am trying to use the Spark API to write data from a file to an ES index.  
My ES Cluster config :  
3 data nodes (with HTTP disabled)  
2 Client nodes (with no data stored on the nodes).

As I understand, the Spark API works only via the HTTP interface. So when I get my process to connect to the Client Nodes, I get an exception that indicates that process does not have access to any of the shards.

Can I use the Spark API in a scenario like mine where I do not have HTTP access to the data nodes.

Thanks

Ramdev

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [November 16, 2015, 4:28pm UTC](https://discuss.elastic.co/t/es-spark-api-writing-to-a-cluster-failes-when-connecting-to-client-node/34697/2 "2015-11-16T16:28:19Z")

</div>

If you are using client nodes, you just need to configure ES-Hadoop [accordingly](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#_network) - see `es.nodes.client.only`.

---

<div class="post-metadata">

**Author:** ![Ramdev\_Wudali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramdev_wudali/32/7894_2.png) [@Ramdev\_Wudali](https://discuss.elastic.co/u/Ramdev_Wudali)\
**Post date:** [November 16, 2015, 6:08pm UTC](https://discuss.elastic.co/t/es-spark-api-writing-to-a-cluster-failes-when-connecting-to-client-node/34697/3 "2015-11-16T18:08:16Z")

</div>

Hi Costin:  
Thanks for the brilliant tip 🙂 I had not looked at all the config  
options available (my bad). That said, I was wondering if there was a  
reason to go only http and not Transport client ?I mean why not enable the  
transport client ?

Thanks much

Ramdev

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [November 16, 2015, 6:28pm UTC](https://discuss.elastic.co/t/es-spark-api-writing-to-a-cluster-failes-when-connecting-to-client-node/34697/4 "2015-11-16T18:28:19Z")

</div>

Transport client is meant for internal use - it might be deprecated in the  
future and it is tied to the version of ES being used. Further more it is  
fairly big.  
REST on the other hand is fully supported, easy to debug/monitor/route,  
provides version isolation and in general, is fully available through  
existing libraries thus ES-Hadoop itself is quite small.

Further more, having hundreds of clients connecting to ES through REST vs  
transport is better (much more scalable) in particular from the perspective  
of a node client.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:27pm UTC](https://discuss.elastic.co/t/es-spark-api-writing-to-a-cluster-failes-when-connecting-to-client-node/34697/5 "2017-07-06T13:27:13Z")

</div>


