# Cannot extract value from entity

**URL:** <https://discuss.elastic.co/t/cannot-extract-value-from-entity/66556>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [November 18, 2016, 3:27pm UTC](https://discuss.elastic.co/t/cannot-extract-value-from-entity/66556 "2016-11-18T15:27:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jspooner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jspooner/32/12984_2.png) [@jspooner](https://discuss.elastic.co/u/jspooner)\
**Post date:** [November 18, 2016, 3:27pm UTC](https://discuss.elastic.co/t/cannot-extract-value-from-entity/66556/1 "2016-11-18T15:27:38Z")

</div>

I've had this script running for some time but it suddenly started throwing an error. The error message looks like my device\_id is in a nested array in my table but after inspecting the table it is not.

```
def esConfig(): Map[String,String] = {
  Map("es.mapping.id" -> "device_id")
}

```

My Table

```
aDF.show()
+-------------------------------------------+                                   
|device_id |
+-------------------------------------------+
|g.40 |
|a.E2 |

```

Running saveToEs throws this error `[ScalaMapFieldExtractor for field [[device_id]]] cannot extract value from entity [class java.lang.String]`

```
EsSpark.saveToEs(aDF.rdd, "production-devices-spark-shell/device", esConfig)

16/11/18 15:20:06 WARN TaskSetManager: Lost task 10.0 in stage 79.0 (TID 3312, ip-10-1-181-210.aws.vrv): org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: [ScalaMapFieldExtractor for field [[device_id]]] cannot extract value from entity [class java.lang.String] | instance [[ds.289d90c249cec7becf3cfde39f0bbe028fcf4ce4]]
	at org.elasticsearch.hadoop.serialization.bulk.AbstractBulkFactory$FieldWriter.write(AbstractBulkFactory.java:97)
	at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.writeTemplate(TemplatedBulk.java:80)
	at org.elasticsearch.hadoop.serialization.bulk.TemplatedBulk.write(TemplatedBulk.java:56)
	at org.elasticsearch.hadoop.rest.RestRepository.writeToIndex(RestRepository.java:159)
	at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:67)
	at org.elasticsearch.spark.rdd.EsSpark$$anonfun$doSaveToEs$1.apply(EsSpark.scala:102)
	at org.elasticsearch.spark.rdd.EsSpark$$anonfun$doSaveToEs$1.apply(EsSpark.scala:102)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
	at org.apache.spark.scheduler.Task.run(Task.scala:86)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
```

---

<div class="post-metadata">

**Author:** ![jspooner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jspooner/32/12984_2.png) [@jspooner](https://discuss.elastic.co/u/jspooner)\
**Post date:** [November 18, 2016, 4:17pm UTC](https://discuss.elastic.co/t/cannot-extract-value-from-entity/66556/2 "2016-11-18T16:17:15Z")

</div>

This issue was resolved by mapping my data set to a Device case class. My script had this operation further up the lineage chain and it appears it must be the last step before handing off the rdd to EsSpark.

```
case class Device(device_id: String)

val esDF = audienceDF.map {
  case Row(device_id: String) => Device(device_id)
}
EsSpark.saveToEs(esDF.rdd, "devices-spark-shell/device", esConfig)
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 16, 2016, 4:18pm UTC](https://discuss.elastic.co/t/cannot-extract-value-from-entity/66556/3 "2016-12-16T16:18:06Z")

</div>

Closed on December 16, 2016, 4:18pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.