# Spark and Elastic node definition issue

**URL:** <https://discuss.elastic.co/t/spark-and-elastic-node-definition-issue/53183>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [June 17, 2016, 6:45pm UTC](https://discuss.elastic.co/t/spark-and-elastic-node-definition-issue/53183 "2016-06-17T18:45:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![gas1474](https://avatars.discourse-cdn.com/v4/letter/g/ac91a4/32.png) [@gas1474](https://discuss.elastic.co/u/gas1474)\
**Post date:** [June 17, 2016, 6:45pm UTC](https://discuss.elastic.co/t/spark-and-elastic-node-definition-issue/53183/1 "2016-06-17T18:45:55Z")

</div>

Hi

i have a VM running elastic elasticsearch-2.1.1 and another RHL running spark spark-1.6.1-bin-hadoop2.6 and using the elastic-hadoop connector elasticsearch-hadoop-2.3.2

if i run spark on the same VM (IP 1.2.3.4) no probleme i can perform some sql using the park-shell  
(although the memeory is not enough)  
if i run the spark from my second machine IP 2.3.4.5 the initialtion work but the resulting command is always zero

> scala\> import org.elasticsearch.spark.sql.\_  
> import org.elasticsearch.spark.sql.\_

> scala\> import org.apache.spark.sql.SQLContext.\_  
> import org.apache.spark.sql.SQLContext.\_

> scala\> import org.apache.spark.sql.SQLContext  
> import org.apache.spark.sql.SQLContext

> scala\> import org.apache.spark.ml.classification.LogisticRegression  
> import org.apache.spark.ml.classification.LogisticRegression

> scala\> val esConfig = Map("pushdown" -\> "true", "es.nodes" -\> "1.2.3.4", "es.port" -\> "9200" , "path" -\> "hfb")  
> esConfig: scala.collection.immutable.Map[String,String] = Map(pushdown -\> true, es.nodes -\> 1.2.3.4, es.port -\> 9200, path -\> orf\_hfb)

> scala\> val df = sqlContext.load("org.elasticsearch.spark.sql", esConfig)  
> warning: there were 1 deprecation warning(s); re-run with -deprecation for details  
> df: org.apache.spark.sql.DataFrame = [@timestamp: timestamp, @version: string, ALARM: string, date: string, host: string, message: string, path: string, tags: string, type: string]

> scala\> sqlContext.sql("CREATE TEMPORARY TABLE myIndex USING org.elasticsearch.spark.sql OPTIONS (resource 'hfb', scroll\_size '20')" )  
> 16/06/17 20:40:59 ERROR NetworkClient: Node [127.0.0.1:9200] failed (Connection refused); no other nodes left - aborting...  
> org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'  
> at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:190)  
> at org.elasticsearch.spark.sql.SchemaUtils$.discoverMappingAsField(SchemaUtils.scala:76)  
> at org.elasticsearch.spark.sql.SchemaUtils$.discoverMapping(SchemaUtils.scala:69)  
> at org.elasticsearch.spark.sql.ElasticsearchRelation.lazySchema$lzycompute(DefaultSource.scala:110)

when i run the same from the local machine (i have installed also the same spark and pacjages) it works

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:24pm UTC](https://discuss.elastic.co/t/spark-and-elastic-node-definition-issue/53183/2 "2017-07-06T13:24:17Z")

</div>