# Spark 2.0.1 one element array issue

**URL:** <https://discuss.elastic.co/t/spark-2-0-1-one-element-array-issue/68829>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [December 13, 2016, 8:33am UTC](https://discuss.elastic.co/t/spark-2-0-1-one-element-array-issue/68829 "2016-12-13T08:33:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mha](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@mha](https://discuss.elastic.co/u/mha)\
**Post date:** [December 13, 2016, 8:33am UTC](https://discuss.elastic.co/t/spark-2-0-1-one-element-array-issue/68829/1 "2016-12-13T08:33:49Z")

</div>

Hi

I'm using elasticsearch-spark-20\_2.10-5.0.1 with spark 2.0.1

I have an object which contains two arrays (mo\_service\_type, mo\_tag) which can contain only one element:

"\_source": {  
"mo\_service\_type": [  
"os",  
"windows",  
"windows\_server",  
"fs"  
],  
"mo\_tag": "glad",  
},

var esConfig:Map[String,String] = Map(  
"es.nodes" -\> "[xyz.eu-west-1.aws.found.io](http://xyz.eu-west-1.aws.found.io)"  
,"es.port" -\> "9200"  
,"es.read.field.as.array.include" -\> "mo\_service\_type,mo\_tag"  
)  
val heartbeatDF = EsSparkSQL.esDF(sqlContext, "heartbeat-\*/heartbeat", esConfig)

this one works:  
%sql  
select mo\_service\_type from hb limit 1

but this one fails:  
%sql  
select mo\_tag from hb limit 1

How can I read this data?

* * *

com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecutionException: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 231.0 failed 4 times, most recent failure: Lost task 0.3 in stage 231.0 (TID 6051, 10.89.251.19): scala.MatchError: glad (of class java.lang.String)  
at org.apache.spark.sql.catalyst.CatalystTypeConverters$ArrayConverter.toCatalystImpl(CatalystTypeConverters.scala:160)  
at org.apache.spark.sql.catalyst.CatalystTypeConverters$ArrayConverter.toCatalystImpl(CatalystTypeConverters.scala:154)  
at ...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 10, 2017, 8:34am UTC](https://discuss.elastic.co/t/spark-2-0-1-one-element-array-issue/68829/2 "2017-01-10T08:34:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.