Multiple inner objects


(Bart Schuller) #1

Hello,

We are evaluating Lucene and in particular Elastic Search to see whether we can replace our existing search engine. One feature it has, which we use, is that it understands hierarchy in content. ElasticSearch has the beginning of similar support, but it looks like it's lacking a way to express certain queries.

An example says more than a thousand words. Below I add two documents representing families and their cars. Neither family owns a gray Toyota, yet both families are returned. The query is missing the constraint that car.make and car.color aren't independent, but should refer to the same car. In the other product, the query would look like:

(((Toyota <IN> make) <AND> (gray <IN> color)) <IN> car)

Would it be possible to add a similar feature to ElasticSearch?

I'm not aware that it's ever been done in a Lucene-based product. This mail to the Solr mailinglist shows how you could index the data such that it becomes possible to query the hierarchy, but care is needed because it uses a concatenation of two fields into one.

Thanks for a very exciting product and thanks or listening,

Regards,

Bart Schuller.

Here's the example as three shell statements:

curl -XPUT 'http://localhost:9200/zonetest/family/1' -d '
{
"name": "Jones",
"car": [
{
"make": "Toyota",
"color": "green"
},
{
"make": "Ford",
"color": "gray"
}
]
}
'

curl -XPUT 'http://localhost:9200/zonetest/family/2' -d '
{
"name": "Gates",
"car": [
{
"make": "Rolls-Royce",
"color": "gray"
},
{
"make": "Toyota",
"color": "blue"
}
]
}
'

curl -XGET 'http://localhost:9200/zonetest/family/_search' -d '
{
"query" : {
"bool" : {
"must" : [
{
"field" : { "car.make" : "Toyota" }
},
{
"field" : { "car.color" : "gray" }
}
]
}
}
}
'

--
Bart.Schuller@gmail.com


(Clinton Gormley) #2

Hiya

An example says more than a thousand words. Below I add two documents
representing families and their cars. Neither family owns a gray
Toyota, yet both families are returned. The query is missing the
constraint that car.make and car.color aren't independent, but should
refer to the same car. In the other product, the query would look
like:

(((Toyota make) (gray color)) car)

Here's an example a search that looks for the words 'foo bar' in
families that own toyotas with colours: red, blue, or gray:

curl -XGET 'http://127.0.0.2:9200/zonetest/family/_search?search_type=dfs_query_then_fetch' -d '
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"must" : [
{
"term" : {
"car.make" : "toyota"
}
},
{
"terms" : {
"car.color" : [
"red",
"blue",
"gray"
]
}
}
]
}
},
"query" : {
"query_string" : {
"query" : "foo bar"
}
}
}
}
}
'


(Bart Schuller) #3

On Apr 19, 2010, at 13:43, Clinton Gormley wrote:

Here's an example a search that looks for the words 'foo bar' in
families that own toyotas with colours: red, blue, or gray:

Well actually, it searches for 'foo bar' in families that own at least one Toyota and that own a car in either of those three colors. What I'm trying to express is that it's the Toyota that should be gray, not just any car like the Rolls.

It's something that would be expressed as a relation (join) if we were using SQL.

curl -XGET 'http://127.0.0.2:9200/zonetest/family/_search?search_type=dfs_query_then_fetch' -d '
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"must" : [
{
"term" : {
"car.make" : "toyota"
}
},
{
"terms" : {
"car.color" : [
"red",
"blue",
"gray"
]
}
}
]
}
},
"query" : {
"query_string" : {
"query" : "foo bar"
}
}
}
}
}
'


(Clinton Gormley) #4

Well actually, it searches for 'foo bar' in families that own at least
one Toyota and that own a car in either of those three colors. What
I'm trying to express is that it's the Toyota that should be gray, not
just any car like the Rolls.

True (and I didn't see your final query that demonstrated the same thing
as I did :slight_smile:

I'm not sure, but I think that currently you'd have to have add a field
in order to achieve this, eg car.make_color


(Shay Banon) #5

Yes, this is not currently possible (easily) with elasticsearch (or other
Lucene based solution) though I do have some ideas (not fully formalized
yet) on how to solve something like this. I hope to work on it soon....

cheers,
shay.banon

On Mon, Apr 19, 2010 at 3:31 PM, Clinton Gormley clinton@iannounce.co.ukwrote:

Well actually, it searches for 'foo bar' in families that own at least
one Toyota and that own a car in either of those three colors. What
I'm trying to express is that it's the Toyota that should be gray, not
just any car like the Rolls.

True (and I didn't see your final query that demonstrated the same thing
as I did :slight_smile:

I'm not sure, but I think that currently you'd have to have add a field
in order to achieve this, eg car.make_color


(Leo) #6

We are hoping to use this feature as well. I am wondering if there is any progress on this subject or how can we add this feature to elasticsearch. Thanks.


(system) #7