# Elasticsearch Hadoop - Issue reading GeoPoint as Dataset

**URL:** <https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [May 30, 2018, 11:54am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865 "2018-05-30T11:54:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![brommer](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@brommer](https://discuss.elastic.co/u/brommer)\
**Post date:** [May 30, 2018, 11:54am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865/1 "2018-05-30T11:54:24Z")

</div>

Hi everyone,

I am using Elasticsearch Hadoop v. 6.2.3 with Spark 2.3.0 trying to read data from Elasticsearch as a Dataset. However, I am receiving following Exception:

```
WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, 172.18.0.2, executor 0): java.lang.IndexOutOfBoundsException: 1
at scala.collection.convert.Wrappers$JListWrapper.productElement(Wrappers.scala:85)
at scala.runtime.ScalaRunTime$$anon$1.next(ScalaRunTime.scala:177)
at org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:251)

```

I am using Scala and have following case class:  
case class MyEntity (  
...  
geometry: Geometry  
...  
)

Geometry looks like this:  
case class Geometry (  
`type`: String,  
coordinates: GeoPoint  
)

And GeoPoint looks like this:  
case class GeoPoint (  
lat: Double,  
lon: Double  
)

The schema of the data looks as follows:  
[info] root  
...  
[info] |-- geometry: struct (nullable = true)  
[info] | |-- coordinates: struct (nullable = true)  
[info] | | |-- lat: double (nullable = true)  
[info] | | |-- lon: double (nullable = true)  
[info] | |-- type: string (nullable = true)  
...

I am reading the data as dataset as follows:  
val myEntities = sqlContext  
.esDF("indexname/doctype")  
.select(  
...  
$"geometry",  
...  
)  
.as[MyEntity]

Without the geometry, I can successfully read the data from Elasticsearch, so it is definitely related to that.

Do you have an idea for the cause of the issue and know how to workaround that issue?

Thanks!

---

<div class="post-metadata">

**Author:** ![brommer](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@brommer](https://discuss.elastic.co/u/brommer)\
**Post date:** [May 30, 2018, 12:12pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865/2 "2018-05-30T12:12:47Z")

</div>

By the way, it seems to be happening within the select statement, not during the conversion to a dataset, since I am receiving the same issue when removing "as[MyEntity]" and just calling "myEntities.show".

---

<div class="post-metadata">

**Author:** ![brommer](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@brommer](https://discuss.elastic.co/u/brommer)\
**Post date:** [May 30, 2018, 12:43pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865/3 "2018-05-30T12:43:16Z")

</div>

This might be related: [https://github.com/elastic/elasticsearch-hadoop/issues/951](https://github.com/elastic/elasticsearch-hadoop/issues/951)

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [May 30, 2018, 7:57pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865/4 "2018-05-30T19:57:11Z")

</div>

It is highly likely that this is related to #951.

Can you post on that issue with your above reproduction for the bug. The more test cases we have on file when I tackle this issue, then the better the solution will be tested before it gets released.

---

<div class="post-metadata">

**Author:** ![brommer](https://avatars.discourse-cdn.com/v4/letter/b/65b543/32.png) [@brommer](https://discuss.elastic.co/u/brommer)\
**Post date:** [May 31, 2018, 6:39am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865/5 "2018-05-31T06:39:25Z")

</div>

> [@brommer](#):
>
> Without the geometry, I can successfully read the data from Elasticsearch, so it is definitely related to that.

Thanks for your quick reply. I have commented on the Github issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 28, 2018, 6:39am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-issue-reading-geopoint-as-dataset/133865/6 "2018-06-28T06:39:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.