# Consequences of excluding fields from \_source

**URL:** https://discuss.elastic.co/t/consequences-of-excluding-fields-from--source/92833
**Category:** Elasticsearch
**Created:** [July 12, 2017, 3:22pm UTC](https://discuss.elastic.co/t/consequences-of-excluding-fields-from--source/92833 "2017-07-12T15:22:45Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![kasscharal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kasscharal/32/20001_2.png) [@kasscharal](https://discuss.elastic.co/u/kasscharal)
#### Post date: [July 12, 2017, 3:22pm UTC](https://discuss.elastic.co/t/consequences-of-excluding-fields-from--source/92833/1 "2017-07-12T15:22:45Z")

</div>

The documentation describes it as an 'expert-only feature' , and it makes it rather clear that it should be avoided. My case is the following: I use the ingest attachment plugin to index and analyze the contents of PDF files. Here is my pipeline definition:

```
{
  "description": "Extract attachment information",
  "processors":[
    {
      "attachment":{
        "field": "data",
        "target_field": "resume",
        "indexed_chars":-1
      }
    }
  ]
}

```

However the above results in the unwanted storing of the files's bytestream under the field attachment, together with the actual contents. In order to save some storage capacity I decided to not store it at all. Here is my mapping:

```
{  
   "_source":{  
      "excludes":[  
         "attachment"
      ]
   },
   "properties":{  
      "attachment":{  
         "type":"text",
         "fields":{  
            "keyword":{  
               "type":"keyword",
               "ignore_above":256
            }
         }
      },
      "resume":{  
         "properties":{  
            "content":{  
               "type":"text",
               "fields":{  
                  "keyword":{  
                     "type":"keyword",
                     "ignore_above":256
                  }
               },
               "analyzer":"some_language"
            }
         }
      }
   }
}

```

This solves my problem (the bytes stream is not returned with my searches), however I'm concerned with the following warning from the [ES docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#include-exclude), on which I'm hoping for some elaboration:

> 'Removing fields from the \_source has similar downsides to disabling \_source, especially the fact that you cannot reindex documents from one Elasticsearch index to another'

How will excluding the attachment from my \_source affect my ability to re-index if needed? And why? Is there another way to do it? Also will the result of the analysis of resume.content field be transferred to the new index?

I hope I'm specific enough, thank you in advance!

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 12, 2017, 3:36pm UTC](https://discuss.elastic.co/t/consequences-of-excluding-fields-from--source/92833/2 "2017-07-12T15:36:06Z")

</div>

It's better IMO to remove the field with an ingest remove processor.  
So the binary BASE64 is not part of the source anymore.

---

<div class="post-metadata">

### Author: ![kasscharal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kasscharal/32/20001_2.png) [@kasscharal](https://discuss.elastic.co/u/kasscharal)
#### Post date: [July 12, 2017, 6:18pm UTC](https://discuss.elastic.co/t/consequences-of-excluding-fields-from--source/92833/3 "2017-07-12T18:18:25Z")

</div>

Dear David,

Thank you for your reply. I will look into it, but just so I understand the full idea, why do you think that this would be better? Does it eliminate the problem of re-indexing?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 12, 2017, 8:37pm UTC](https://discuss.elastic.co/t/consequences-of-excluding-fields-from--source/92833/4 "2017-07-12T20:37:24Z")

</div>

Yes. As it does not alter the `_source` field.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 9, 2017, 8:37pm UTC](https://discuss.elastic.co/t/consequences-of-excluding-fields-from--source/92833/5 "2017-08-09T20:37:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
