# Missing filter on fields which are nested objects **URL:** \ **Category:** Elasticsearch\ **Created:** [June 6, 2013, 8:10am UTC](https://discuss.elastic.co/t/missing-filter-on-fields-which-are-nested-objects/12293 "2013-06-06T08:10:19Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Abhishek\_Sanoujam](https://avatars.discourse-cdn.com/v4/letter/a/3da27b/32.png) [@Abhishek\_Sanoujam](https://discuss.elastic.co/u/Abhishek_Sanoujam)\ **Post date:** [June 6, 2013, 8:10am UTC](https://discuss.elastic.co/t/missing-filter-on-fields-which-are-nested-objects/12293/1 "2013-06-06T08:10:19Z")
Hi The "missing" filter doesn't seem to work on fields which are nested objects. Its supposed to return documents which don't have the specified field, and it works for "flattened" attributes. But if the field itself is a nested object, it doesn't work. Though it works again on attributes on the nested object itself. Is this a bug or expected behavior? How can i find documents which are missing some fields which are nested objects? To illustrate ([https://gist.github.com/abhi-sanoujam/5720015](https://gist.github.com/abhi-sanoujam/5720015)): - Put 3 documents - Search for "missing" a field which is top level attribute - WORKS - Search for "missing" a field which is a nested object - DOESN'T WORK - Search for "missing" a field within a nested object - WORKS curl -XPUT [http://localhost:9200/test/test/1](http://localhost:9200/test/test/1) -d '{ "a" : "A1", "b" : { "b\_a" : "b\_a 1" } }' curl -XPUT [http://localhost:9200/test/test/2](http://localhost:9200/test/test/2) -d '{ "abc" : "abc 1" }' curl -XPUT [http://localhost:9200/test/test/3](http://localhost:9200/test/test/3) -d '{ "a" : "A3", "b" : { "b\_b" : "b\_b 1" } }' curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)" Returns all docs\* \* _1) Search for missing field which is not a nested object - WORKS_ curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)" -d '{ "filter" : { "missing" : { "field" : "a" } } }' _Returns document with id=2 (which has only field "abc") [EXPECTED]_ _2) Search for missing field which is a nested object - DOESN'T WORK_ curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)" -d '{ "filter" : { "missing" : { "field" : "b" } } }' _Returns all docs. **Expected to have returned only doc with id=2** , as that's the only document that's missing the attribute "b"._ [NOT EXPECTED] _3) Search for missing field within nested objects - WORKS_ curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)" -d '{ "filter" : { "missing" : { "field" : "b.b\_a" } } }' _Returns doc with id 2 and 3 [EXPECTED]_ curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)" -d '{ "filter" : { "missing" : { "field" : "b.b\_b" } } }' \*Returns doc with id 1 and 2 \*_[EXPECTED]_ ## -- Cheers, Abhishek -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [June 6, 2013, 8:55am UTC](https://discuss.elastic.co/t/missing-filter-on-fields-which-are-nested-objects/12293/2 "2013-06-06T08:55:28Z")
Hey, currently it is not possible to apply the missing filter on an object (because the object is not really indexed compared to the fields inside of the object). The current workaround would be to call missing on a field inside of the object. As this is not really userfriendly, I have created an issue for that. > > > The missing filter is not working, when the field expected to be missing is an o…bject. Consider this sample > > \`\`\` sh > curl -X DELETE http://localhost:9200/test > > curl -XPUT 'http://localhost:9200/test/test/3?refresh=true' -d '{ > "foo" : { > "bar" : "baz" > } > }' > > curl -XPOST "http://localhost:9200/test/\_search?pretty" -d '{ > "filter" : { > "missing" : { > "field" : "foo" > } > } > }' > \`\`\` > > This could be implemented by checking the mapping for the type to be queried and create a bool filter full of missing filters, when the mapping shows an object type. > > This feature also supports wildcards notation, for example \`obj1.obj2.field\_pre\*\`. --Alex On Thu, Jun 6, 2013 at 10:10 AM, Abhishek Sanoujam [abhi.sanoujam@gmail.com](mailto:abhi.sanoujam@gmail.com)wrote: > Hi > > The "missing" filter doesn't seem to work on fields which are nested > objects. Its supposed to return documents which don't have the specified > field, and it works for "flattened" attributes. But if the field itself is > a nested object, it doesn't work. Though it works again on attributes on > the nested object itself. > > Is this a bug or expected behavior? How can i find documents which are > missing some fields which are nested objects? > > To illustrate ([doesnt\_work.sh · GitHub](https://gist.github.com/abhi-sanoujam/5720015)): > > - Put 3 documents > - Search for "missing" a field which is top level attribute - WORKS > - Search for "missing" a field which is a nested object - DOESN'T WORK > - Search for "missing" a field within a nested object - WORKS > > curl -XPUT [http://localhost:9200/test/test/1](http://localhost:9200/test/test/1) -d '{ > "a" : "A1", > "b" : { > "b\_a" : "b\_a 1" > } > }' > > curl -XPUT [http://localhost:9200/test/test/2](http://localhost:9200/test/test/2) -d '{ > "abc" : "abc 1" > }' > > curl -XPUT [http://localhost:9200/test/test/3](http://localhost:9200/test/test/3) -d '{ > "a" : "A3", > "b" : { > "b\_b" : "b\_b 1" > } > }' > > curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)"[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty) > Returns all docs\* > \* > _1) Search for missing field which is not a nested object - WORKS_ > curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)"[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)-d '{ > "filter" : { > "missing" : { > "field" : "a" > } > } > }' > _Returns document with id=2 (which has only field "abc") [EXPECTED]_ > > _2) Search for missing field which is a nested object - DOESN'T WORK_ > curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)"[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)-d '{ > "filter" : { > "missing" : { > "field" : "b" > } > } > }' > > _Returns all docs. **Expected to have returned only doc with id=2** , as > that's the only document that's missing the attribute "b"._ [NOT EXPECTED] > > _3) Search for missing field within nested objects - WORKS_ > curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)"[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)-d '{ > "filter" : { > "missing" : { > "field" : "b.b\_a" > } > } > }' > _Returns doc with id 2 and 3 [EXPECTED]_ > > curl -XPOST "[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)"[http://localhost:9200/test/\_search?pretty](http://localhost:9200/test/_search?pretty)-d '{ > "filter" : { > "missing" : { > "field" : "b.b\_b" > } > } > }' > \*Returns doc with id 1 and 2 \*_[EXPECTED]_ > > ## -- > > Cheers, > Abhishek > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Abhishek\_Sanoujam](https://avatars.discourse-cdn.com/v4/letter/a/3da27b/32.png) [@Abhishek\_Sanoujam](https://discuss.elastic.co/u/Abhishek_Sanoujam)\ **Post date:** [June 6, 2013, 8:59am UTC](https://discuss.elastic.co/t/missing-filter-on-fields-which-are-nested-objects/12293/3 "2013-06-06T08:59:00Z")
Thanks for the input Alex. Hope the issue gets fixed pretty soon 🙂 For my use case, I don't really have a choice as the fields inside the object are also dynamic and there's no fixed "attribute" in the nested object that I can use to decide the existence. Looks like i'll have to workaround and update the document by adding a field to mark existence/non-existence of certain nested objects. On 6/6/13 2:25 PM, Alexander Reelsen wrote: > Hey, > > currently it is not possible to apply the missing filter on an object > (because the object is not really indexed compared to the fields > inside of the object). > The current workaround would be to call missing on a field inside of > the object. > As this is not really userfriendly, I have created an issue for that. > > [missing/exists filters should also work for objects · Issue #3141 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3141) > > --Alex > > On Thu, Jun 6, 2013 at 10:10 AM, Abhishek Sanoujam > \<[abhi.sanoujam@gmail.com](mailto:abhi.sanoujam@gmail.com) [mailto:abhi.sanoujam@gmail.com](mailto:abhi.sanoujam@gmail.com)\> wrote: > > ``` > Hi > > The "missing" filter doesn't seem to work on fields which are > nested objects. Its supposed to return documents which don't have > the specified field, and it works for "flattened" attributes. But > if the field itself is a nested object, it doesn't work. Though it > works again on attributes on the nested object itself. > > Is this a bug or expected behavior? How can i find documents which > are missing some fields which are nested objects? > > To illustrate (https://gist.github.com/abhi-sanoujam/5720015): > - Put 3 documents > - Search for "missing" a field which is top level attribute - WORKS > - Search for "missing" a field which is a nested object - DOESN'T WORK > - Search for "missing" a field within a nested object - WORKS > > curl -XPUT http://localhost:9200/test/test/1 -d '{ > "a" : "A1", > "b" : { > "b_a" : "b_a 1" > } > }' > > curl -XPUT http://localhost:9200/test/test/2 -d '{ > "abc" : "abc 1" > }' > > curl -XPUT http://localhost:9200/test/test/3 -d '{ > "a" : "A3", > "b" : { > "b_b" : "b_b 1" > } > }' > > curl -XPOST "http://localhost:9200/test/_search?pretty" > > Returns all docs* > * > *1) Search for missing field which is not a nested object - WORKS* > curl -XPOST "http://localhost:9200/test/_search?pretty" > -d '{ > "filter" : { > "missing" : { > "field" : "a" > } > } > }' > *Returns document with id=2 (which has only field "abc") [EXPECTED]* > > *2) Search for missing field which is a nested object - DOESN'T WORK* > curl -XPOST "http://localhost:9200/test/_search?pretty" > -d '{ > "filter" : { > "missing" : { > "field" : "b" > } > } > }' > > *Returns all docs. **Expected to have returned only doc with > id=2**, as that's the only document that's missing the attribute > "b".* [NOT EXPECTED] > > *3) Search for missing field within nested objects - WORKS* > curl -XPOST "http://localhost:9200/test/_search?pretty" > -d '{ > "filter" : { > "missing" : { > "field" : "b.b_a" > } > } > }' > *Returns doc with id 2 and 3 [EXPECTED]* > > curl -XPOST "http://localhost:9200/test/_search?pretty" > -d '{ > "filter" : { > "missing" : { > "field" : "b.b_b" > } > } > }' > *Returns doc with id 1 and 2 **[EXPECTED]* > > -- > --------- > Cheers, > Abhishek > > -- > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, > send an email to elasticsearch+unsubscribe@googlegroups.com > . > For more options, visit https://groups.google.com/groups/opt_out. > > ``` > > -- > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ## -- Cheers, Abhishek -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:32am UTC](https://discuss.elastic.co/t/missing-filter-on-fields-which-are-nested-objects/12293/4 "2017-07-06T02:32:39Z")