Exists Filter doesn't work with objects


(MarcusLongmuir) #1

I understand that this functionality has not been extended to objects and
this is not a bug.

I posted this on a thread in which the OP found an alternative solution,
but unfortunately I cannot use the same workaround.

https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ

Using the following structures which are representative of the problem, but
are not my structures.

[
{
"User" : "Marcus",
"Content" : {
"Content" : "This is a post",
"Title" : "The title of this post"
},
"Media" : {
"Header" : "http://.../images/one.jpg"
}
},
{
"User" : "Marcus",
"Content" : {
"Content" : "This is another post, but without a header",
"Title" : "The title of this other post"
}
}
]

I'd like to be able to find all documents with a "Media" object. The
workaround the OP used in the linked thread was to use one of the core keys
in the "Media" object, but I can't use any of its keys because they differ
between documents.

Has anyone solved this problem before?

Thanks


(Shay Banon) #2

Can you add such a field? it can be a simple thing like "media" : true.

On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir marcuslongmuir@me.comwrote:

I understand that this functionality has not been extended to objects and
this is not a bug.

I posted this on a thread in which the OP found an alternative solution,
but unfortunately I cannot use the same workaround.

https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ

Using the following structures which are representative of the problem,
but are not my structures.

[
{
"User" : "Marcus",
"Content" : {
"Content" : "This is a post",
"Title" : "The title of this post"
},
"Media" : {
"Header" : "http://.../images/one.jpg"
}
},
{
"User" : "Marcus",
"Content" : {
"Content" : "This is another post, but without a header",
"Title" : "The title of this other post"
}
}
]

I'd like to be able to find all documents with a "Media" object. The
workaround the OP used in the linked thread was to use one of the core keys
in the "Media" object, but I can't use any of its keys because they differ
between documents.

Has anyone solved this problem before?

Thanks


(MarcusLongmuir) #3

The "media" key I used in my example is a user-specified key in both
insertion and retrieval so I would require keys for all objects of this
type.

I see that using a boolean value for a representative key (an escape
character prepended to the relevant key) would solve the search problem,
but the results are returned directly in an API in which the "search only"
keys are not appropriate.

I would have to manually remove all keys which began with the escape
character which would be intensive as currently the source string is not
evaluated to a JSON object in my application before transmission.

I can also see how this can be achieved without evaluating the JSON source
of the document by iterating through the string and recognising the unique
set of characters that represent the existence of a "search only" key and
removing it, but this is also reasonably intensive.

Is this the best option in your opinion?

Thanks

On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote:

Can you add such a field? it can be a simple thing like "media" : true.

On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir marcuslongmuir@me.comwrote:

I understand that this functionality has not been extended to objects and
this is not a bug.

I posted this on a thread in which the OP found an alternative solution,
but unfortunately I cannot use the same workaround.

https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ

Using the following structures which are representative of the problem,
but are not my structures.

[
{
"User" : "Marcus",
"Content" : {
"Content" : "This is a post",
"Title" : "The title of this post"
},
"Media" : {
"Header" : "http://.../images/one.jpg"
}
},
{
"User" : "Marcus",
"Content" : {
"Content" : "This is another post, but without a header",
"Title" : "The title of this other post"
}
}
]

I'd like to be able to find all documents with a "Media" object. The
workaround the OP used in the linked thread was to use one of the core keys
in the "Media" object, but I can't use any of its keys because they differ
between documents.

Has anyone solved this problem before?

Thanks


(Shay Banon) #4

Yea, currently, you will have to add your own field that said "exists" in
the Media object. We could, potentially, add a flag in the mapping of
object to automatically add an "object._exists" field for you? Does it make
sense?

On Wed, Apr 25, 2012 at 8:36 PM, marcuslongmuir marcuslongmuir@me.comwrote:

The "media" key I used in my example is a user-specified key in both
insertion and retrieval so I would require keys for all objects of this
type.

I see that using a boolean value for a representative key (an escape
character prepended to the relevant key) would solve the search problem,
but the results are returned directly in an API in which the "search only"
keys are not appropriate.

I would have to manually remove all keys which began with the escape
character which would be intensive as currently the source string is not
evaluated to a JSON object in my application before transmission.

I can also see how this can be achieved without evaluating the JSON source
of the document by iterating through the string and recognising the unique
set of characters that represent the existence of a "search only" key and
removing it, but this is also reasonably intensive.

Is this the best option in your opinion?

Thanks

On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote:

Can you add such a field? it can be a simple thing like "media" : true.

On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir marcuslongmuir@me.comwrote:

I understand that this functionality has not been extended to objects
and this is not a bug.

I posted this on a thread in which the OP found an alternative solution,
but unfortunately I cannot use the same workaround.

https://groups.google.com/forum/?fromgroups#!searchin/
elasticsearch/marcuslongmuir/**elasticsearch/_Sf_lt5ABA/2W**x1jg4vFUJhttps://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ

Using the following structures which are representative of the problem,
but are not my structures.

[
{
"User" : "Marcus",
"Content" : {
"Content" : "This is a post",
"Title" : "The title of this post"
},
"Media" : {
"Header" : "http://.../images/one.jpg"
}
},
{
"User" : "Marcus",
"Content" : {
"Content" : "This is another post, but without a header",
"Title" : "The title of this other post"
}
}
]

I'd like to be able to find all documents with a "Media" object. The
workaround the OP used in the linked thread was to use one of the core keys
in the "Media" object, but I can't use any of its keys because they differ
between documents.

Has anyone solved this problem before?

Thanks


(MarcusLongmuir) #5

I think for completeness and for intuitive use the exists filter should be
usable with object keys. I was surprised to find that this hasn't been
discussed much before as I can see quite generic use cases.

Is there any way I can include this functionality in my own setup or will I
have to wait for a new build?

Thanks for your help Shay

On Friday, 27 April 2012 10:06:46 UTC+1, kimchy wrote:

Yea, currently, you will have to add your own field that said "exists" in
the Media object. We could, potentially, add a flag in the mapping of
object to automatically add an "object._exists" field for you? Does it make
sense?

On Wed, Apr 25, 2012 at 8:36 PM, marcuslongmuir marcuslongmuir@me.comwrote:

The "media" key I used in my example is a user-specified key in both
insertion and retrieval so I would require keys for all objects of this
type.

I see that using a boolean value for a representative key (an escape
character prepended to the relevant key) would solve the search problem,
but the results are returned directly in an API in which the "search only"
keys are not appropriate.

I would have to manually remove all keys which began with the escape
character which would be intensive as currently the source string is not
evaluated to a JSON object in my application before transmission.

I can also see how this can be achieved without evaluating the JSON
source of the document by iterating through the string and recognising the
unique set of characters that represent the existence of a "search only"
key and removing it, but this is also reasonably intensive.

Is this the best option in your opinion?

Thanks

On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote:

Can you add such a field? it can be a simple thing like "media" : true.

On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir marcuslongmuir@me.comwrote:

I understand that this functionality has not been extended to objects
and this is not a bug.

I posted this on a thread in which the OP found an alternative
solution, but unfortunately I cannot use the same workaround.

https://groups.google.com/forum/?fromgroups#!searchin/
elasticsearch/marcuslongmuir/**elasticsearch/_Sf_lt5ABA/2W**x1jg4vFUJhttps://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ

Using the following structures which are representative of the problem,
but are not my structures.

[
{
"User" : "Marcus",
"Content" : {
"Content" : "This is a post",
"Title" : "The title of this post"
},
"Media" : {
"Header" : "http://.../images/one.jpg"
}
},
{
"User" : "Marcus",
"Content" : {
"Content" : "This is another post, but without a header",
"Title" : "The title of this other post"
}
}
]

I'd like to be able to find all documents with a "Media" object. The
workaround the OP used in the linked thread was to use one of the core keys
in the "Media" object, but I can't use any of its keys because they differ
between documents.

Has anyone solved this problem before?

Thanks


(Shay Banon) #6

At the end of the day, exists filter works on what gets indexed, nothing
gets indexed for an object, except for its fields. You can add a field to
the object you care about in the json yourself (like exists or something
similar) and then check on it.

On Sat, Apr 28, 2012 at 1:52 PM, marcuslongmuir marcuslongmuir@me.comwrote:

I think for completeness and for intuitive use the exists filter should be
usable with object keys. I was surprised to find that this hasn't been
discussed much before as I can see quite generic use cases.

Is there any way I can include this functionality in my own setup or will
I have to wait for a new build?

Thanks for your help Shay

On Friday, 27 April 2012 10:06:46 UTC+1, kimchy wrote:

Yea, currently, you will have to add your own field that said "exists" in
the Media object. We could, potentially, add a flag in the mapping of
object to automatically add an "object._exists" field for you? Does it make
sense?

On Wed, Apr 25, 2012 at 8:36 PM, marcuslongmuir marcuslongmuir@me.comwrote:

The "media" key I used in my example is a user-specified key in both
insertion and retrieval so I would require keys for all objects of this
type.

I see that using a boolean value for a representative key (an escape
character prepended to the relevant key) would solve the search problem,
but the results are returned directly in an API in which the "search only"
keys are not appropriate.

I would have to manually remove all keys which began with the escape
character which would be intensive as currently the source string is not
evaluated to a JSON object in my application before transmission.

I can also see how this can be achieved without evaluating the JSON
source of the document by iterating through the string and recognising the
unique set of characters that represent the existence of a "search only"
key and removing it, but this is also reasonably intensive.

Is this the best option in your opinion?

Thanks

On Wednesday, 25 April 2012 16:55:59 UTC+1, kimchy wrote:

Can you add such a field? it can be a simple thing like "media" : true.

On Mon, Apr 23, 2012 at 6:24 PM, marcuslongmuir marcuslongmuir@me.comwrote:

I understand that this functionality has not been extended to objects
and this is not a bug.

I posted this on a thread in which the OP found an alternative
solution, but unfortunately I cannot use the same workaround.

https://groups.google.com/forum/?fromgroups#!searchin/elasti
csearch/marcuslongmuir/elasticsearch/_Sf_lt5ABA/2Wx1jg4vFUJhttps://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/marcuslongmuir/elasticsearch/__Sf_lt5ABA/2W_x1jg4vFUJ

Using the following structures which are representative of the
problem, but are not my structures.

[
{
"User" : "Marcus",
"Content" : {
"Content" : "This is a post",
"Title" : "The title of this post"
},
"Media" : {
"Header" : "http://.../images/one.jpg"
}
},
{
"User" : "Marcus",
"Content" : {
"Content" : "This is another post, but without a header",
"Title" : "The title of this other post"
}
}
]

I'd like to be able to find all documents with a "Media" object. The
workaround the OP used in the linked thread was to use one of the core keys
in the "Media" object, but I can't use any of its keys because they differ
between documents.

Has anyone solved this problem before?

Thanks


(system) #7