# Scala.MatchError: when trying to read data from elasticsearch

**URL:** <https://discuss.elastic.co/t/scala-matcherror-when-trying-to-read-data-from-elasticsearch/28712>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [September 5, 2015, 2:41am UTC](https://discuss.elastic.co/t/scala-matcherror-when-trying-to-read-data-from-elasticsearch/28712 "2015-09-05T02:41:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sandeep\_gunnam](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sandeep\_gunnam](https://discuss.elastic.co/u/sandeep_gunnam)\
**Post date:** [September 5, 2015, 2:41am UTC](https://discuss.elastic.co/t/scala-matcherror-when-trying-to-read-data-from-elasticsearch/28712/1 "2015-09-05T02:41:01Z")

</div>

Hi  
I am running Spark 1.3.1 in local mode and my scala version is 2.10. I have some data indexed into elasticsearch and trying to read that using **_hiveContext.esDF_** method. It works for other schemas of data but fails for the one shown below  
{  
"Name": "ken1",  
"businessCategory": "BMS",  
"conditions": [{  
"category": "bb",  
"diagCode": "cc",  
"diagCodeType": "D",  
"name": "Blood Pressure",  
"severity": "Medium"  
}],  
"id": 1,  
"organizationId": "NXP"  
}

When i try to execute sql query on this data like "select \* from /" i get an exception like shown below  
scala.MatchError: (StringType,[[bb,cc,D,Blood Pressure,Medium]]) (of class scala.Tuple2)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1.apply(JsonRDD.scala:451)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1.apply(JsonRDD.scala:451)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1$$anonfun$apply$7.apply(JsonRDD.scala:486)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1$$anonfun$apply$7.apply(JsonRDD.scala:482)  
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)  
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1.apply(JsonRDD.scala:482)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1.apply(JsonRDD.scala:451)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1$$anonfun$apply$7.apply(JsonRDD.scala:486)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1$$anonfun$apply$7.apply(JsonRDD.scala:482)  
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)  
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1.apply(JsonRDD.scala:482)  
at org.apache.spark.sql.json.JsonRDD$$anonfun$org$apache$spark$sql$json$JsonRDD$$valWriter$1$1.apply(JsonRDD.scala:451)  
at org.apache.spark.sql.json.JsonRDD$.rowToJSON(JsonRDD.scala:491)  
at org.apache.spark.sql.DataFrame$$anonfun$toJSON$1$$anon$1.next(DataFrame.scala:1233)  
at org.apache.spark.sql.DataFrame$$anonfun$toJSON$1$$anon$1.next(DataFrame.scala:1230)  
at scala.collection.Iterator$class.foreach(Iterator.scala:727)  
at org.apache.spark.sql.DataFrame$$anonfun$toJSON$1$$anon$1.foreach(DataFrame.scala:1230)  
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)  
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)  
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)  
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)  
at org.apache.spark.sql.DataFrame$$anonfun$toJSON$1$$anon$1.to(DataFrame.scala:1230)  
at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)  
at org.apache.spark.sql.DataFrame$$anonfun$toJSON$1$$anon$1.toBuffer(DataFrame.scala:1230)  
at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)  
at org.apache.spark.sql.DataFrame$$anonfun$toJSON$1$$anon$1.toArray(DataFrame.scala:1230)  
at org.apache.spark.rdd.RDD$$anonfun$17.apply(RDD.scala:813)  
at org.apache.spark.rdd.RDD$$anonfun$17.apply(RDD.scala:813)  
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1498)  
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1498)  
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)  
at org.apache.spark.scheduler.Task.run(Task.scala:64)  
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:745)  
I see no elasticsearch-spark code in the stack trace. The code is exactly failing at the line **_hiveContext.sql(query).toJSON.collect().mkString(",")_** in my code. Do you see something wrong with the kind of data i indexed? I tried to read and write the same json using spark shell (using sqlcontext to read json and convert that to jsonRdd) which worked fine.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [September 5, 2015, 8:53pm UTC](https://discuss.elastic.co/t/scala-matcherror-when-trying-to-read-data-from-elasticsearch/28712/2 "2015-09-05T20:53:46Z")

</div>

Why are you using `hivecontext`? The data is read from Elasticsearch not Hive. Use `SqlContext` and the proper methods.

By the looks of it, Spark chokes on parsing the JSON - unclear though why that is.

P.S. Please use some formatting next time - your post is unreadable..

---

<div class="post-metadata">

**Author:** ![sandeep\_gunnam](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sandeep\_gunnam](https://discuss.elastic.co/u/sandeep_gunnam)\
**Post date:** [September 10, 2015, 5:52am UTC](https://discuss.elastic.co/t/scala-matcherror-when-trying-to-read-data-from-elasticsearch/28712/3 "2015-09-10T05:52:31Z")

</div>

Thanks for your response costin and sorry about the formatting (not sure how to correct it though). I used hivecontext as that object is already available in the code i am integrating. As you suggested I will try with sqlContext.

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [September 10, 2015, 8:54am UTC](https://discuss.elastic.co/t/scala-matcherror-when-trying-to-read-data-from-elasticsearch/28712/4 "2015-09-10T08:54:22Z")

</div>

`HiveContext` extends `SqlContext` however internally it is designed to work on Hive (as the name suggests). Using `SqlContext` is more appropriate since there's no Hive underneath. Also since you have `HiveContext` you should be able to get access to the underlying `SqlContext` or you can simply create a new one through `new SqlContext(SparkContext)`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:27pm UTC](https://discuss.elastic.co/t/scala-matcherror-when-trying-to-read-data-from-elasticsearch/28712/5 "2017-07-06T13:27:33Z")

</div>