# Retrieve \_parent field with spark

**URL:** <https://discuss.elastic.co/t/retrieve--parent-field-with-spark/53941>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [June 25, 2016, 11:12am UTC](https://discuss.elastic.co/t/retrieve--parent-field-with-spark/53941 "2016-06-25T11:12:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pawel\_Chabierski](https://avatars.discourse-cdn.com/v4/letter/p/ecc23a/32.png) [@Pawel\_Chabierski](https://discuss.elastic.co/u/Pawel_Chabierski)\
**Post date:** [June 25, 2016, 11:12am UTC](https://discuss.elastic.co/t/retrieve--parent-field-with-spark/53941/1 "2016-06-25T11:12:03Z")

</div>

Hi,

I have problem with retrieving \_parent field from elasticsearch using pyspark. Rdd does not contains \_parent field if I specify that field in fields. My Code:

```
es_query = {

    "fields": ["_parent", "_source"]

}

es_read_conf = {
     "es.nodes" : "localhost",
     "es.resource" : "crm/event",
     "es.nodes.wan.only": "true",
     "es.query": json.dumps(es_query),
      "es.read.metadata": "true"
}

 es_rdd = SparkContext().newAPIHadoopRDD(
     inputFormatClass="org.elasticsearch.hadoop.mr.EsInputFormat",
     keyClass="org.apache.hadoop.io.NullWritable",
     valueClass="org.elasticsearch.hadoop.mr.LinkedMapWritable",
     conf=es_read_conf)

    print(es_rdd.first())

```

Rdd looks like:

```
(u'wn4U76ggQKKEKjBNEqiVyg', {u'isp': None, u'tags': None, u'url': None, u'ip': None, u'website_id': 4, u'_metadata': {u'_score': 0.0, u'_type': u'event', u'_id': u'wn4U76ggQKKEKjBNEqiVyg', u'_index': u'crm'}, u'create_timestamp': 1462300665000, u'fields': {}, u'type': u'met_scenario_condition', u'additional_data': {u'block_id': 261, u'scenario_id': 13, u'block_name': u'Warunek wej\u015bciowy', u'action_type': u'mail', u'scenario_name': u'SP Ostatnio przegl\u0105dane', u'action_id': 261}})

```

When I use this query in elasticsearch-hammer \_parent field is present in response.

I've tried every version, including alpha version. What's wrong?

---

<div class="post-metadata">

**Author:** ![Pawel\_Chabierski](https://avatars.discourse-cdn.com/v4/letter/p/ecc23a/32.png) [@Pawel\_Chabierski](https://discuss.elastic.co/u/Pawel_Chabierski)\
**Post date:** [July 2, 2016, 4:48pm UTC](https://discuss.elastic.co/t/retrieve--parent-field-with-spark/53941/2 "2016-07-02T16:48:14Z")

</div>

I found that by default fields not present in mapping are deleted from result. It can be disabled by setting es.read.unmapped.fields.ignore to false.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:24pm UTC](https://discuss.elastic.co/t/retrieve--parent-field-with-spark/53941/3 "2017-07-06T13:24:04Z")

</div>