# Strange behaviour of the \_source.excludes mapping
**URL:** https://discuss.elastic.co/t/strange-behaviour-of-the--source-excludes-mapping/8702
**Category:** Elasticsearch
**Created:** [August 10, 2012, 8:55am UTC](https://discuss.elastic.co/t/strange-behaviour-of-the--source-excludes-mapping/8702 "2012-08-10T08:55:17Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Virgile\_Devaux\_2](https://discuss.elastic.co/u/Virgile_Devaux_2)
#### Post date: [August 10, 2012, 8:55am UTC](https://discuss.elastic.co/t/strange-behaviour-of-the--source-excludes-mapping/8702/1 "2012-08-10T08:55:17Z")
If I start ES with the basic (almost empty) default mapping, I can issue
the following commands:
[localhost:~]\> curl -XPUT '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)' -d '{
"user" : "kimchy",
"post\_date" : "2009-11-15T14:12:12",
"message" : "trying out Elastic Search",
"emptyObject" : {}
}'
{"ok":true,"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1}
[localhost:~]\> curl -XGET '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)'
{"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1,"exists":true,
"\_source" : {
"user" : "kimchy",
"post\_date" : "2009-11-15T14:12:12",
"message" : "trying out Elastic Search",
"emptyObject" : {}
}}
Notice that the empty object is returned as it has been sent.
But if I set the default mapping as following:
{
"_default_" : {
"\_source" : {
"excludes" : ["test"]
}
}
}
then
[localhost:~]\> curl -XGET '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)'
{"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1,"exists":true,
"\_source" : {"message":"trying out Elastic
Search","user":"kimchy","post\_date":"2009-11-15T14:12:12"}}
the empty object has been removed form the \_source!!!
I've made other tests showing that it works only on root fields.
Could someone confirm and explain?
---
### Author:  [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)
#### Post date: [August 10, 2012, 6:37pm UTC](https://discuss.elastic.co/t/strange-behaviour-of-the--source-excludes-mapping/8702/2 "2012-08-10T18:37:58Z")
When the "excludes" or "includes" lists are present, source goes through
parse/filter/serialize process before it's saved as a \_source field. As a
result of this process, source text is first parsed into a map, then this
map is filtered into another map, and, finally, the new map is serialized
back into the text form. If source is already in the correct format
and "excludes" and "includes" lists are not present, the source is stored
as is.
While elements are copied from one map into another, the filtering process
omits excluded fields as well as empty fields[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/xcontent/support/XContentMapValues.java#L183](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/xcontent/support/XContentMapValues.java#L183).
I would guess that it's done to remove higher level elements that lost all
nested element as a result of the filtering. Moreover, since empty elements
are not indexed anyway, it makes little sense to keep them in the source.
On Friday, August 10, 2012 4:55:17 AM UTC-4, Virgile Devaux wrote:
> If I start ES with the basic (almost empty) default mapping, I can issue
> the following commands:
> [localhost:~]\> curl -XPUT '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)' -d '{
> "user" : "kimchy",
> "post\_date" : "2009-11-15T14:12:12",
> "message" : "trying out Elastic Search",
> "emptyObject" : {}
> }'
> {"ok":true,"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1}
> [localhost:~]\> curl -XGET '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)'
> {"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1,"exists":true,
> "\_source" : {
> "user" : "kimchy",
> "post\_date" : "2009-11-15T14:12:12",
> "message" : "trying out Elastic Search",
> "emptyObject" : {}
> }}
>
> Notice that the empty object is returned as it has been sent.
> But if I set the default mapping as following:
> {
> "_default_" : {
> "\_source" : {
> "excludes" : ["test"]
> }
> }
> }
>
> then
> [localhost:~]\> curl -XGET '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)'
> {"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1,"exists":true,
> "\_source" : {"message":"trying out Elastic
> Search","user":"kimchy","post\_date":"2009-11-15T14:12:12"}}
>
> the empty object has been removed form the \_source!!!
> I've made other tests showing that it works only on root fields.
> Could someone confirm and explain?
--
---
### Author:  [@Virgile\_Devaux\_2](https://discuss.elastic.co/u/Virgile_Devaux_2)
#### Post date: [August 13, 2012, 8:32am UTC](https://discuss.elastic.co/t/strange-behaviour-of-the--source-excludes-mapping/8702/3 "2012-08-13T08:32:35Z")
Ok thanks, Igor for this very in depth response.
I'm not sure that an empty field, even not indexed, is the same as no field
at all, but it is not a problem for us for now. May be I'll make a pull
request for this sometime.
Thanks again.
Le vendredi 10 août 2012 20:37:58 UTC+2, Igor Motov a écrit :
> When the "excludes" or "includes" lists are present, source goes through
> parse/filter/serialize process before it's saved as a \_source field. As a
> result of this process, source text is first parsed into a map, then this
> map is filtered into another map, and, finally, the new map is serialized
> back into the text form. If source is already in the correct format
> and "excludes" and "includes" lists are not present, the source is stored
> as is.
>
> While elements are copied from one map into another, the filtering process
> omits excluded fields as well as empty fields[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/xcontent/support/XContentMapValues.java#L183](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/common/xcontent/support/XContentMapValues.java#L183).
> I would guess that it's done to remove higher level elements that lost all
> nested element as a result of the filtering. Moreover, since empty elements
> are not indexed anyway, it makes little sense to keep them in the source.
>
> On Friday, August 10, 2012 4:55:17 AM UTC-4, Virgile Devaux wrote:
>
> > If I start ES with the basic (almost empty) default mapping, I can issue
> > the following commands:
> > [localhost:~]\> curl -XPUT '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)' -d '{
> > "user" : "kimchy",
> > "post\_date" : "2009-11-15T14:12:12",
> > "message" : "trying out Elastic Search",
> > "emptyObject" : {}
> > }'
> > {"ok":true,"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1}
> > [localhost:~]\> curl -XGET '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)'
> > {"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1,"exists":true,
> > "\_source" : {
> > "user" : "kimchy",
> > "post\_date" : "2009-11-15T14:12:12",
> > "message" : "trying out Elastic Search",
> > "emptyObject" : {}
> > }}
> >
> > Notice that the empty object is returned as it has been sent.
> > But if I set the default mapping as following:
> > {
> > "_default_" : {
> > "\_source" : {
> > "excludes" : ["test"]
> > }
> > }
> > }
> >
> > then
> > [localhost:~]\> curl -XGET '[http://localhost:9200/twitter/tweet/1](http://localhost:9200/twitter/tweet/1)'
> > {"\_index":"twitter","\_type":"tweet","\_id":"1","\_version":1,"exists":true,
> > "\_source" : {"message":"trying out Elastic
> > Search","user":"kimchy","post\_date":"2009-11-15T14:12:12"}}
> >
> > the empty object has been removed form the \_source!!!
> > I've made other tests showing that it works only on root fields.
> > Could someone confirm and explain?
--
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:16am UTC](https://discuss.elastic.co/t/strange-behaviour-of-the--source-excludes-mapping/8702/4 "2017-07-06T03:16:37Z")