# Avoid es.mapping.id field duplication

**URL:** <https://discuss.elastic.co/t/avoid-es-mapping-id-field-duplication/66851>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [November 22, 2016, 12:04pm UTC](https://discuss.elastic.co/t/avoid-es-mapping-id-field-duplication/66851 "2016-11-22T12:04:01Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rishav\_Rohit1](https://avatars.discourse-cdn.com/v4/letter/r/e19b73/32.png) [@Rishav\_Rohit1](https://discuss.elastic.co/u/Rishav_Rohit1)\
**Post date:** [November 22, 2016, 12:04pm UTC](https://discuss.elastic.co/t/avoid-es-mapping-id-field-duplication/66851/1 "2016-11-22T12:04:02Z")

</div>

Hi,

I am using ElasticSearch Hadoop (2.4.0) connector for storing documents in ES. I have "record\_id" field in my document which is used as \_id field using below configuration setting  
conf.set("[es.mapping.id](http://es.mapping.id)", "record\_id");  
This record\_id is stored twice, first in "\_id" field and then in "record\_id" field.  
What configuration should I use to avoid duplication of this field?

Thanks.

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [November 28, 2016, 4:32pm UTC](https://discuss.elastic.co/t/avoid-es-mapping-id-field-duplication/66851/2 "2016-11-28T16:32:05Z")

</div>

@Rishav_Rohit1 You can also specify which fields should be ignored from the document body during serialization by using the `es.mapping.exclude` setting:

```auto
sc.makeRDD(Map("id" -> "1", "name" -> "Jimmy") :: Nil)
    .saveToEs("test/test", Map("es.mapping.id" -> "id", "es.mapping.exclude" -> "id"))

```

This tells the connector to use the "id" field as the document's id, but to ignore it when creating the body of the document. The resulting output is as follows:

```auto
{
    "took":39,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1.0,
        "hits":[
            {
                "_index":"test",
                "_type":"test",
                "_id":"1",
                "_score":1.0,
                "_source":{"name":"Jimmy"}
            }
        ]
    }
}

```

---

<div class="post-metadata">

**Author:** ![Rishav\_Rohit1](https://avatars.discourse-cdn.com/v4/letter/r/e19b73/32.png) [@Rishav\_Rohit1](https://discuss.elastic.co/u/Rishav_Rohit1)\
**Post date:** [November 30, 2016, 10:00am UTC](https://discuss.elastic.co/t/avoid-es-mapping-id-field-duplication/66851/3 "2016-11-30T10:00:37Z")

</div>

Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 28, 2016, 10:01am UTC](https://discuss.elastic.co/t/avoid-es-mapping-id-field-duplication/66851/4 "2016-12-28T10:01:29Z")

</div>

Closed on December 28, 2016, 10:01am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.