# Elasticsearch return empty list of objects to spark

**URL:** https://discuss.elastic.co/t/elasticsearch-return-empty-list-of-objects-to-spark/288052
**Category:** Elasticsearch
**Created:** [October 30, 2021, 5:43am UTC](https://discuss.elastic.co/t/elasticsearch-return-empty-list-of-objects-to-spark/288052 "2021-10-30T05:43:42Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![mohsen\_mortezapour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohsen_mortezapour/32/96554_2.png) [@mohsen\_mortezapour](https://discuss.elastic.co/u/mohsen_mortezapour)
#### Post date: [October 30, 2021, 5:43am UTC](https://discuss.elastic.co/t/elasticsearch-return-empty-list-of-objects-to-spark/288052/1 "2021-10-30T05:43:42Z")

</div>

I have a problem with **Elasticsearch**.

I want to read data from **Elasticsearch** index with **Pyspark**. my data look like as below:

```auto
user_id: 123,
features: {
    hashtags: [
        {
            text: "hello",
            count: 2
        },
        {
            text: "world",
            count: 1
        }
    ]
}
...

```

I have a problem with **Elasticsearch**.

I want to read data from **Elasticsearch** index with **Pyspark**. my data look like as below:

```auto
user_id: 123,
features: {
    hashtags: [
        {
            text: "hello",
            count: 2
        },
        {
            text: "world",
            count: 1
        }
    ]
}
...

```

and when data loaded it seems **Elasticsearch** return empty list of objects. my dataframe after read is look like as below:

```auto
+----------+-------------------+
| features| user_id|
+----------+-------------------+
|{[{}, {}]}| 123|
| {[{}]}| 384|
| {[{}]}| 94|
|{[{}, {}]}| 880|
+----------+-------------------+

```

I read data from elastic with using this configuration:

```auto
tweets = sqlContext.read.format("org.elasticsearch.spark.sql") \
    .option("es.nodes", "localhost") \
    .option("es.port", "9200") \
    .option("es.read.field.as.array.include", "features.hashtags")\
    .option("es.read.field.include", "user_id, features.hashtags")\
    .option("es.resource", "twitter")\
    .load().limit(10)

```

can you help me for resolve it?

---

<div class="post-metadata">

### Author: ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)
#### Post date: [November 9, 2021, 10:58pm UTC](https://discuss.elastic.co/t/elasticsearch-return-empty-list-of-objects-to-spark/288052/2 "2021-11-09T22:58:29Z")

</div>

We're now actively discussing this at [Elasticsearch return empty list of objects to spark · Issue #1784 · elastic/elasticsearch-hadoop · GitHub](https://github.com/elastic/elasticsearch-hadoop/issues/1784).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 7, 2021, 10:59pm UTC](https://discuss.elastic.co/t/elasticsearch-return-empty-list-of-objects-to-spark/288052/3 "2021-12-07T22:59:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.