# Exists Filter doesn't work with objects **URL:** \ **Category:** Elasticsearch\ **Created:** [April 23, 2012, 3:24pm UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444 "2012-04-23T15:24:11Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![MarcusLongmuir](https://avatars.discourse-cdn.com/v4/letter/m/13edae/32.png) [@MarcusLongmuir](https://discuss.elastic.co/u/MarcusLongmuir)\ **Post date:** [April 23, 2012, 3:24pm UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444/1 "2012-04-23T15:24:11Z")
I understand that this functionality has not been extended to objects and this is not a bug. I posted this on a thread in which the OP found an alternative solution, but unfortunately I cannot use the same workaround. [https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/\_\_Sf\_lt5ABA/2W\_x1jg4vFUJ](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ) Using the following structures which are representative of the problem, but are not my structures. [ { "User" : "Marcus", "Content" : { "Content" : "This is a post", "Title" : "The title of this post" }, "Media" : { "Header" : "http://.../images/one.jpg" } }, { "User" : "Marcus", "Content" : { "Content" : "This is another post, but without a header", "Title" : "The title of this other post" } } ] I'd like to be able to find all documents with a "Media" object. The workaround the OP used in the linked thread was to use one of the core keys in the "Media" object, but I can't use any of its keys because they differ between documents. Has anyone solved this problem before? Thanks ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 25, 2012, 3:55pm UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444/2 "2012-04-25T15:55:59Z")
Can you add such a field? it can be a simple thing like "media" : true. On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > I understand that this functionality has not been extended to objects and > this is not a bug. > > I posted this on a thread in which the OP found an alternative solution, > but unfortunately I cannot use the same workaround. > > [Redirecting to Google Groups](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ) > > Using the following structures which are representative of the problem, > but are not my structures. > > [ > { > "User" : "Marcus", > "Content" : { > "Content" : "This is a post", > "Title" : "The title of this post" > }, > "Media" : { > "Header" : "http://.../images/one.jpg" > } > }, > { > "User" : "Marcus", > "Content" : { > "Content" : "This is another post, but without a header", > "Title" : "The title of this other post" > } > } > ] > > I'd like to be able to find all documents with a "Media" object. The > workaround the OP used in the linked thread was to use one of the core keys > in the "Media" object, but I can't use any of its keys because they differ > between documents. > > Has anyone solved this problem before? > > Thanks ---
**Author:** ![MarcusLongmuir](https://avatars.discourse-cdn.com/v4/letter/m/13edae/32.png) [@MarcusLongmuir](https://discuss.elastic.co/u/MarcusLongmuir)\ **Post date:** [April 25, 2012, 5:36pm UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444/3 "2012-04-25T17:36:48Z")
The "media" key I used in my example is a user-specified key in both insertion and retrieval so I would require keys for all objects of this type. I see that using a boolean value for a representative key (an escape character prepended to the relevant key) would solve the search problem, but the results are returned directly in an API in which the "search only" keys are not appropriate. I would have to manually remove all keys which began with the escape character which would be intensive as currently the source string is not evaluated to a JSON object in my application before transmission. I can also see how this can be achieved without evaluating the JSON source of the document by iterating through the string and recognising the unique set of characters that represent the existence of a "search only" key and removing it, but this is also reasonably intensive. Is this the best option in your opinion? Thanks On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote: > Can you add such a field? it can be a simple thing like "media" : true. > > On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > > > I understand that this functionality has not been extended to objects and > > this is not a bug. > > > > I posted this on a thread in which the OP found an alternative solution, > > but unfortunately I cannot use the same workaround. > > > > [Redirecting to Google Groups](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ) > > > > Using the following structures which are representative of the problem, > > but are not my structures. > > > > [ > > { > > "User" : "Marcus", > > "Content" : { > > "Content" : "This is a post", > > "Title" : "The title of this post" > > }, > > "Media" : { > > "Header" : "http://.../images/one.jpg" > > } > > }, > > { > > "User" : "Marcus", > > "Content" : { > > "Content" : "This is another post, but without a header", > > "Title" : "The title of this other post" > > } > > } > > ] > > > > I'd like to be able to find all documents with a "Media" object. The > > workaround the OP used in the linked thread was to use one of the core keys > > in the "Media" object, but I can't use any of its keys because they differ > > between documents. > > > > Has anyone solved this problem before? > > > > Thanks ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 27, 2012, 9:06am UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444/4 "2012-04-27T09:06:46Z")
Yea, currently, you will have to add your own field that said "exists" in the Media object. We could, potentially, add a flag in the mapping of object to automatically add an "object.\_exists" field for you? Does it make sense? On Wed, Apr 25, 2012 at 8:36 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > The "media" key I used in my example is a user-specified key in both > insertion and retrieval so I would require keys for all objects of this > type. > > I see that using a boolean value for a representative key (an escape > character prepended to the relevant key) would solve the search problem, > but the results are returned directly in an API in which the "search only" > keys are not appropriate. > > I would have to manually remove all keys which began with the escape > character which would be intensive as currently the source string is not > evaluated to a JSON object in my application before transmission. > > I can also see how this can be achieved without evaluating the JSON source > of the document by iterating through the string and recognising the unique > set of characters that represent the existence of a "search only" key and > removing it, but this is also reasonably intensive. > > Is this the best option in your opinion? > > Thanks > > On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote: > > > Can you add such a field? it can be a simple thing like "media" : true. > > > > On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > > > > > I understand that this functionality has not been extended to objects > > > and this is not a bug. > > > > > > I posted this on a thread in which the OP found an alternative solution, > > > but unfortunately I cannot use the same workaround. > > > > > > [https://groups.google.com/\*\*forum/?fromgroups#!searchin/](https://groups.google.com/**forum/?fromgroups#!searchin/)\*\* > > > elasticsearch/marcuslongmuir/\*\*elasticsearch/\__Sf\_lt5ABA/2W_\*\*x1jg4vFUJ[https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/\_\_Sf\_lt5ABA/2W\_x1jg4vFUJ](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ) > > > > > > Using the following structures which are representative of the problem, > > > but are not my structures. > > > > > > [ > > > { > > > "User" : "Marcus", > > > "Content" : { > > > "Content" : "This is a post", > > > "Title" : "The title of this post" > > > }, > > > "Media" : { > > > "Header" : "http://.../images/one.jpg" > > > } > > > }, > > > { > > > "User" : "Marcus", > > > "Content" : { > > > "Content" : "This is another post, but without a header", > > > "Title" : "The title of this other post" > > > } > > > } > > > ] > > > > > > I'd like to be able to find all documents with a "Media" object. The > > > workaround the OP used in the linked thread was to use one of the core keys > > > in the "Media" object, but I can't use any of its keys because they differ > > > between documents. > > > > > > Has anyone solved this problem before? > > > > > > Thanks ---
**Author:** ![MarcusLongmuir](https://avatars.discourse-cdn.com/v4/letter/m/13edae/32.png) [@MarcusLongmuir](https://discuss.elastic.co/u/MarcusLongmuir)\ **Post date:** [April 28, 2012, 10:52am UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444/5 "2012-04-28T10:52:52Z")
I think for completeness and for intuitive use the exists filter should be usable with object keys. I was surprised to find that this hasn't been discussed much before as I can see quite generic use cases. Is there any way I can include this functionality in my own setup or will I have to wait for a new build? Thanks for your help Shay On Friday, 27 April 2012 10:06:46 UTC+1, kimchy wrote: > Yea, currently, you will have to add your own field that said "exists" in > the Media object. We could, potentially, add a flag in the mapping of > object to automatically add an "object.\_exists" field for you? Does it make > sense? > > On Wed, Apr 25, 2012 at 8:36 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > > > The "media" key I used in my example is a user-specified key in both > > insertion and retrieval so I would require keys for all objects of this > > type. > > > > I see that using a boolean value for a representative key (an escape > > character prepended to the relevant key) would solve the search problem, > > but the results are returned directly in an API in which the "search only" > > keys are not appropriate. > > > > I would have to manually remove all keys which began with the escape > > character which would be intensive as currently the source string is not > > evaluated to a JSON object in my application before transmission. > > > > I can also see how this can be achieved without evaluating the JSON > > source of the document by iterating through the string and recognising the > > unique set of characters that represent the existence of a "search only" > > key and removing it, but this is also reasonably intensive. > > > > Is this the best option in your opinion? > > > > Thanks > > > > On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote: > > > > > Can you add such a field? it can be a simple thing like "media" : true. > > > > > > On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > > > > > > > I understand that this functionality has not been extended to objects > > > > and this is not a bug. > > > > > > > > I posted this on a thread in which the OP found an alternative > > > > solution, but unfortunately I cannot use the same workaround. > > > > > > > > [https://groups.google.com/\*\*forum/?fromgroups#!searchin/](https://groups.google.com/**forum/?fromgroups#!searchin/)\*\* > > > > elasticsearch/marcuslongmuir/\*\*elasticsearch/\__Sf\_lt5ABA/2W_\*\*x1jg4vFUJ[https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/\_\_Sf\_lt5ABA/2W\_x1jg4vFUJ](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ) > > > > > > > > Using the following structures which are representative of the problem, > > > > but are not my structures. > > > > > > > > [ > > > > { > > > > "User" : "Marcus", > > > > "Content" : { > > > > "Content" : "This is a post", > > > > "Title" : "The title of this post" > > > > }, > > > > "Media" : { > > > > "Header" : "http://.../images/one.jpg" > > > > } > > > > }, > > > > { > > > > "User" : "Marcus", > > > > "Content" : { > > > > "Content" : "This is another post, but without a header", > > > > "Title" : "The title of this other post" > > > > } > > > > } > > > > ] > > > > > > > > I'd like to be able to find all documents with a "Media" object. The > > > > workaround the OP used in the linked thread was to use one of the core keys > > > > in the "Media" object, but I can't use any of its keys because they differ > > > > between documents. > > > > > > > > Has anyone solved this problem before? > > > > > > > > Thanks ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 29, 2012, 5:32pm UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444/6 "2012-04-29T17:32:39Z")
At the end of the day, exists filter works on what gets indexed, nothing gets indexed for an object, except for its fields. You can add a field to the object you care about in the json yourself (like exists or something similar) and then check on it. On Sat, Apr 28, 2012 at 1:52 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > I think for completeness and for intuitive use the exists filter should be > usable with object keys. I was surprised to find that this hasn't been > discussed much before as I can see quite generic use cases. > > Is there any way I can include this functionality in my own setup or will > I have to wait for a new build? > > Thanks for your help Shay > > On Friday, 27 April 2012 10:06:46 UTC+1, kimchy wrote: > > > Yea, currently, you will have to add your own field that said "exists" in > > the Media object. We could, potentially, add a flag in the mapping of > > object to automatically add an "object.\_exists" field for you? Does it make > > sense? > > > > On Wed, Apr 25, 2012 at 8:36 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > > > > > The "media" key I used in my example is a user-specified key in both > > > insertion and retrieval so I would require keys for all objects of this > > > type. > > > > > > I see that using a boolean value for a representative key (an escape > > > character prepended to the relevant key) would solve the search problem, > > > but the results are returned directly in an API in which the "search only" > > > keys are not appropriate. > > > > > > I would have to manually remove all keys which began with the escape > > > character which would be intensive as currently the source string is not > > > evaluated to a JSON object in my application before transmission. > > > > > > I can also see how this can be achieved without evaluating the JSON > > > source of the document by iterating through the string and recognising the > > > unique set of characters that represent the existence of a "search only" > > > key and removing it, but this is also reasonably intensive. > > > > > > Is this the best option in your opinion? > > > > > > Thanks > > > > > > On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote: > > > > > > > Can you add such a field? it can be a simple thing like "media" : true. > > > > > > > > On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir [marcuslongmuir@me.com](mailto:marcuslongmuir@me.com)wrote: > > > > > > > > > I understand that this functionality has not been extended to objects > > > > > and this is not a bug. > > > > > > > > > > I posted this on a thread in which the OP found an alternative > > > > > solution, but unfortunately I cannot use the same workaround. > > > > > > > > > > [https://groups.google.com/\*\*foru\*\*m/?fromgroups#!searchin/\*\*elasti](https://groups.google.com/ **foru** m/?fromgroups#!searchin/**elasti)\*\* > > > > > csearch/marcuslongmuir/ **elastic** search/\__Sf\_lt5ABA/2W_ **x1jg4vFU** J[https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/\_\_Sf\_lt5ABA/2W\_x1jg4vFUJ](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ) > > > > > > > > > > Using the following structures which are representative of the > > > > > problem, but are not my structures. > > > > > > > > > > [ > > > > > { > > > > > "User" : "Marcus", > > > > > "Content" : { > > > > > "Content" : "This is a post", > > > > > "Title" : "The title of this post" > > > > > }, > > > > > "Media" : { > > > > > "Header" : "http://.../images/one.jpg" > > > > > } > > > > > }, > > > > > { > > > > > "User" : "Marcus", > > > > > "Content" : { > > > > > "Content" : "This is another post, but without a header", > > > > > "Title" : "The title of this other post" > > > > > } > > > > > } > > > > > ] > > > > > > > > > > I'd like to be able to find all documents with a "Media" object. The > > > > > workaround the OP used in the linked thread was to use one of the core keys > > > > > in the "Media" object, but I can't use any of its keys because they differ > > > > > between documents. > > > > > > > > > > Has anyone solved this problem before? > > > > > > > > > > Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:30am UTC](https://discuss.elastic.co/t/exists-filter-doesnt-work-with-objects/7444/7 "2017-07-06T03:30:37Z")