# Field 'tags' is backed by an array but the associated Spark Schema does not reflect this

**URL:** <https://discuss.elastic.co/t/field-tags-is-backed-by-an-array-but-the-associated-spark-schema-does-not-reflect-this/194049>\
**Category:** Elasticsearch\
**Created:** [August 6, 2019, 3:58pm UTC](https://discuss.elastic.co/t/field-tags-is-backed-by-an-array-but-the-associated-spark-schema-does-not-reflect-this/194049 "2019-08-06T15:58:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![basha5](https://avatars.discourse-cdn.com/v4/letter/b/e5b9ba/32.png) [@basha5](https://discuss.elastic.co/u/basha5)\
**Post date:** [August 6, 2019, 3:58pm UTC](https://discuss.elastic.co/t/field-tags-is-backed-by-an-array-but-the-associated-spark-schema-does-not-reflect-this/194049/1 "2019-08-06T15:58:42Z")

</div>

Hi,  
I am trying to read the data from index in ES using pyspark. I am able to load the data into the dataframe. Used this code  
reader = spark.read.format("es").option("es.read.metadata", True).option("es.port","9200").option("es.nodes", "[http://xx.x.x.xxx](http://xx.x.x.xxx)").option("es.read.field.as.array.include","XRAM,XCPU,count,message")

df = reader.load("webserverstats")

df.printSchema()

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/d/2d762ddbc73e0cc8653531ca3a3fefc446f248ae.png) . Its schema is there. Later on this error occured.  
new\_df=df.toPandas()  
new\_df.show()  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/2/5246b52a82c68ec1b9a7484771ca303ef20dab02.png) .

Could anybody help resolve this issue.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 3, 2019, 3:58pm UTC](https://discuss.elastic.co/t/field-tags-is-backed-by-an-array-but-the-associated-spark-schema-does-not-reflect-this/194049/2 "2019-09-03T15:58:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
