Score of matched (nested) documents?

polyfractal · August 5, 2012, 11:52pm

I believe the answer is "no", but wanted to check:

Is there a way to return the individual score of the nested documents that
match a query? I'm performing a query on nested docs which returns
appropriate parents, but I would like to know which nested child scored the
highest for the query. The nested score_mode simply tells me that "one of
the children scored well", but not which one. Is there any way to sort or
find the score of the nested children?

Barring that, I'm not certain of the best way to structure my data. Should
I arrange the data in two types and denormalize so that one type has a
"foreign key" (to use relational db terminology) that points at the other
type, which would then require several more queries to access the
individual documents?

Thanks!
-Zach

polyfractal · August 6, 2012, 1:04pm

Looks like I found my answer here:

https://groups.google.com/d/topic/elasticsearch/1_1zXSeRsxQ/discussion

Guess I'll start down the denormalization path. IF anyone has some
comments or opinions on the best way to structure relationships in ES, I'd
love to hear. Is the general sentiment to not model relationships and
simply retrieve IDs which you use through another, non-ES mechanism to find
(say, query a database) ?

Related, what's the performance of simply looking up a specific document by
it's ID? Is it O(n)? Does ES use something like b-trees for document ID
lookups?

Thanks!
-Zach

On Sunday, August 5, 2012 7:52:54 PM UTC-4, Zachary Tong wrote:

I believe the answer is "no", but wanted to check:

Is there a way to return the individual score of the nested documents that
match a query? I'm performing a query on nested docs which returns
appropriate parents, but I would like to know which nested child scored the
highest for the query. The nested score_mode simply tells me that "one of
the children scored well", but not which one. Is there any way to sort or
find the score of the nested children?

Barring that, I'm not certain of the best way to structure my data.
Should I arrange the data in two types and denormalize so that one type
has a "foreign key" (to use relational db terminology) that points at the
other type, which would then require several more queries to access the
individual documents?

Thanks!
-Zach

kimchy · August 7, 2012, 9:40pm

There is no option to get score of specific nested doc, though it has been requested to be able to return "only" the nested doc that matched the query, in which case we can possibly return the score with it. This is a very "up in the air" feature currently, not slated (or even designed) yet.

Regarding the "get" API pert, its not a btree, it will probably be a bit faster then a btree. Thats because there is a bloom filter associated with each segment, so getting a doc will almost all times find the relevant segment, and then within that segment (a segment is part of an index) its more similar to an interval based skip list.

On Aug 6, 2012, at 3:04 PM, Zachary Tong zacharyjtong@gmail.com wrote:

Looks like I found my answer here:

https://groups.google.com/d/topic/elasticsearch/1_1zXSeRsxQ/discussion

Guess I'll start down the denormalization path. IF anyone has some comments or opinions on the best way to structure relationships in ES, I'd love to hear. Is the general sentiment to not model relationships and simply retrieve IDs which you use through another, non-ES mechanism to find (say, query a database) ?

Related, what's the performance of simply looking up a specific document by it's ID? Is it O(n)? Does ES use something like b-trees for document ID lookups?

Thanks!
-Zach

On Sunday, August 5, 2012 7:52:54 PM UTC-4, Zachary Tong wrote:
I believe the answer is "no", but wanted to check:

Is there a way to return the individual score of the nested documents that match a query? I'm performing a query on nested docs which returns appropriate parents, but I would like to know which nested child scored the highest for the query. The nested score_mode simply tells me that "one of the children scored well", but not which one. Is there any way to sort or find the score of the nested children?

Barring that, I'm not certain of the best way to structure my data. Should I arrange the data in two types and denormalize so that one type has a "foreign key" (to use relational db terminology) that points at the other type, which would then require several more queries to access the individual documents?

Thanks!
-Zach

polyfractal · August 7, 2012, 10:20pm

Good to know, thanks for the detailed response Shay!

-Zach

On Tuesday, August 7, 2012 5:40:09 PM UTC-4, kimchy wrote:

There is no option to get score of specific nested doc, though it has been
requested to be able to return "only" the nested doc that matched the
query, in which case we can possibly return the score with it. This is a
very "up in the air" feature currently, not slated (or even designed) yet.

Regarding the "get" API pert, its not a btree, it will probably be a bit
faster then a btree. Thats because there is a bloom filter associated with
each segment, so getting a doc will almost all times find the relevant
segment, and then within that segment (a segment is part of an index) its
more similar to an interval based skip list.

On Aug 6, 2012, at 3:04 PM, Zachary Tong <zachar...@gmail.com<javascript:>>
wrote:

Looks like I found my answer here:

https://groups.google.com/d/topic/elasticsearch/1_1zXSeRsxQ/discussion

Guess I'll start down the denormalization path. IF anyone has some
comments or opinions on the best way to structure relationships in ES, I'd
love to hear. Is the general sentiment to not model relationships and
simply retrieve IDs which you use through another, non-ES mechanism to find
(say, query a database) ?

Related, what's the performance of simply looking up a specific document
by it's ID? Is it O(n)? Does ES use something like b-trees for document
ID lookups?

Thanks!
-Zach

On Sunday, August 5, 2012 7:52:54 PM UTC-4, Zachary Tong wrote:

I believe the answer is "no", but wanted to check:

Is there a way to return the individual score of the nested documents
that match a query? I'm performing a query on nested docs which returns
appropriate parents, but I would like to know which nested child scored the
highest for the query. The nested score_mode simply tells me that "one of
the children scored well", but not which one. Is there any way to sort or
find the score of the nested children?

Barring that, I'm not certain of the best way to structure my data.
Should I arrange the data in two types and denormalize so that one type
has a "foreign key" (to use relational db terminology) that points at the
other type, which would then require several more queries to access the
individual documents?

Thanks!
-Zach

btiernay · January 18, 2013, 2:54pm

It's been a little time since this post was created I'm curious to know if
there are any plans to add this feature (child filtering w/ score) any time
in the future?

Thanks!

--

btiernay · January 18, 2013, 3:01pm

I suppose this is the following ticket?

github.com/elastic/elasticsearch

Expose matching nested docs to the query and sort

opened 10:43AM - 08 Oct 11 UTC

closed 03:06PM - 10 May 13 UTC

clintongormley

Hiya I have a use case for nested docs which, currently, can not be supported e…asily. For instance, have a look at this doc: ``` { "location" : { "lat" : "8.31667", "lon" : "52.38333" }, "parent_ids" : [ "2862926", "2921044" ], "contexts" : [ { "context" : "/en", "rank" : 2, "tokens" : [ { "tokens" : [ "steinbrink" ] }, { "tokens" : [ "steinbrink", "germany" ] } ], "label" : { "short" : "Steinbrink", "long" : "Steinbrink, Germany" } }, { "context" : "/de", "rank" : 3, "tokens" : [ { "tokens" : [ "steinbrink" ] }, { "tokens" : [ "steinbrink", "deutschland" ] } ], "label" : { "short" : "Steinbrink", "long" : "Steinbrink, Deutschland" } } ] } ``` `contexts` and `tokens` are both nested docs. I would like to run query like: - filter by `contexts.context == '/de'` - sort by: - `contexts.context.rank desc` - `contexts.label.short asc` but of course I have no way of pulling out the single matching context and using the values in that sub-doc. I could use a script sort like the horribly inefficient: ``` { "sort" : [ { "params" : { "name" : "/de" }, "_script" : { "script" : "for (context: contexts) { if (context.context == name) { return context.rank }}", "order" : "desc", "type" : "number" } }, { "_script" : { "params" : { "name" : "/de" }, "script" : "for (context: contexts) { if (context.context == name) { return context.label.short }}", "order" : "asc", "type" : "string" } } ] } ``` Or I may want to do: - filter by `context=='/de'` - find doc with tokens that best match `stein*` - return the `tokens` that match best as a field Or have a slightly different doc structure: ``` "tokens" : [ { "label": "Steinbrink", "tokens" : [ "steinbrink" ] }, { "label": "Steinbrink, Germany", "tokens" : [ "steinbrink, germany" ] }, ], ``` and return the `label` associated with the `tokens` that best match. Even with a script, I couldn't find the best matching `tokens`. In this example, I only have two contexts, and so could have changed the doc structure to something like: ``` contexts: { en: {....}, de: {....} } ``` But I have other use cases where I have hundreds of contexts, which would result in thousands of fields. Would it be possible to add functionality to: - expose the best matching child (or children, with the best matching first in an array) in a `_scope` so we could do something like: "sort": [ { "_child.myscope[0].rank": "desc" }, { "_child.myscope[0].label.short": "asc"} ], "fields": [ "_child.otherscope[0],tokens.label" ] - return the scores for matching children in a `_scope` eg: { "children": { "myscope": [5,3,0,2,0], "otherscope": [0,1,0] } } Not sure how feasible this is, and if it is feasible, there is probably a much more elegant way of doing it. thanks clint

On Friday, 18 January 2013 09:54:55 UTC-5, btiernay wrote:

It's been a little time since this post was created I'm curious to know if
there are any plans to add this feature (child filtering w/ score) any time
in the future?

Thanks!

--

Nova · November 12, 2013, 10:09am

Example: we search an address "Dien Bien Phu", but we get ["Dien Bien Phu", "Hung Vuong", Thanh Khe"], Can we rank score for every nested document in this case?
I think if so, "Dien Bien Phu" in nested document will have highest score.

How do you think about this ?