# Source field exclusion seems to be storing data anyway

**URL:** <https://discuss.elastic.co/t/source-field-exclusion-seems-to-be-storing-data-anyway/21487>\
**Category:** Elasticsearch\
**Created:** [January 5, 2015, 5:30pm UTC](https://discuss.elastic.co/t/source-field-exclusion-seems-to-be-storing-data-anyway/21487 "2015-01-05T17:30:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Randy\_McCluer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/randy_mccluer/32/990_2.png) [@Randy\_McCluer](https://discuss.elastic.co/u/Randy_McCluer)\
**Post date:** [January 5, 2015, 5:30pm UTC](https://discuss.elastic.co/t/source-field-exclusion-seems-to-be-storing-data-anyway/21487/1 "2015-01-05T17:30:36Z")

</div>

I have some sensitive data that I want excluded from source, but indexed. I  
am using "\_source": { "excludes": ["field1"] }, and everything seems to be  
working just as expected with the source docs coming back without field1.  
If I update the mapping to not exclude field1, the docs still return  
without field1 as well. However, if I restart the service, they start  
coming back with field1 in the doc, indicating that the data was being  
stored all along.

All of the documentation I've found indicates that the excluded fields are  
removed at write-time. My situation leads me to believe that this isn't the  
case. Can someone tell me if this is the expected behavior or a bug to be  
filed? I'd really hate to have to go down the Solr-style route of declaring  
all of my fields individually.

TIA

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/dd420599-3a17-43a5-b5f1-ab552e910a94%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/dd420599-3a17-43a5-b5f1-ab552e910a94%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Randy\_McCluer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/randy_mccluer/32/990_2.png) [@Randy\_McCluer](https://discuss.elastic.co/u/Randy_McCluer)\
**Post date:** [January 14, 2015, 7:22pm UTC](https://discuss.elastic.co/t/source-field-exclusion-seems-to-be-storing-data-anyway/21487/2 "2015-01-14T19:22:05Z")

</div>

I posted this over the holidays, so I figured I'd bump it. Anyone else ever  
seen this behavior?

On Monday, January 5, 2015 at 11:30:36 AM UTC-6, Randy McCluer wrote:

> I have some sensitive data that I want excluded from source, but indexed.  
> I am using "\_source": { "excludes": ["field1"] }, and everything seems to  
> be working just as expected with the source docs coming back without  
> field1. If I update the mapping to not exclude field1, the docs still  
> return without field1 as well. However, if I restart the service, they  
> start coming back with field1 in the doc, indicating that the data was  
> being stored all along.
> 
> All of the documentation I've found indicates that the excluded fields are  
> removed at write-time. My situation leads me to believe that this isn't the  
> case. Can someone tell me if this is the expected behavior or a bug to be  
> filed? I'd really hate to have to go down the Solr-style route of declaring  
> all of my fields individually.
> 
> TIA

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/44a91939-4526-42a6-8085-ae632be40812%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/44a91939-4526-42a6-8085-ae632be40812%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Benjamin\_Gathmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benjamin_gathmann/32/7561_2.png) [@Benjamin\_Gathmann](https://discuss.elastic.co/u/Benjamin_Gathmann)\
**Post date:** [December 23, 2015, 2:45pm UTC](https://discuss.elastic.co/t/source-field-exclusion-seems-to-be-storing-data-anyway/21487/3 "2015-12-23T14:45:09Z")

</div>

Hi, I have just opened a similar question on this topic. From all the posts I have read, Elasticsearch does not seem to behave very reliable in respect to excluding fields or paths from \_source and being indexed.: [Exclude complete path from indexing and \_source](https://discuss.elastic.co/t/exclude-complete-path-from-indexing-and--source/37854/2)  
I am curious to see answers to this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:29pm UTC](https://discuss.elastic.co/t/source-field-exclusion-seems-to-be-storing-data-anyway/21487/4 "2017-07-05T23:29:04Z")

</div>


