BUG?: faceting on nested-documents (with scope) doesn't take filter on parent-docs into account

Geert_Jan_Brits · December 21, 2012, 11:16am

I'm banging my head against the wall on trying on faceting on
nested-documents with:

a filter on the nested documents
a filter on the 'parent' documents

To me the query (at end of post) should just work. Explanation:

a queryfilter is created limiting all hits AND facets to only match
against parent-docs with field rec=80
a queryfilter is created limiting all hits AND facets to only match
against parent-docs which have a nested-doc with ddp.ddpindex = "b" (this
filter is defined as scope: 'scope_ddp_exists')

Then I'd like to facet on nested field 'ddp.price' using just defined scope
'scope_ddp_exists'.
To my understanding (since facets should already be contrained by 'rec=80')
this should return nested docs:

of which the parent is contrained to rec=80
of which the nested-doc itself has the field ddpindex='b'

However, the filter " rec=80" isn't taken into account at all.

What's wrong with this filter? It is as if 'scope_ddp_exists' is placed
directly under the global-scope, instead of taking queryfilters set under
scope 'main' into account?
If this is intended, is there any other way to construct the query to facet
on scope 'scope_ddp_exists' while still taking the other (query)-filters
into account?

{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"and": [
{
"term": {"rec": 80}
},
{
"nested":{
"_scope": "scope_ddp_exists",
"path":"ddp",
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"term":{"ddp.ddpindex":"b"}
}
}
}
}
}
]
}
}
},
"facets": {
"priceslider": {
"terms": {
"field": "ddp.price"
},
"scope": "scope_ddp_exists"
}
}
}

Thanks,
Geert-Jan

--

Geert_Jan_Brits · December 21, 2012, 11:17am

On Friday, December 21, 2012 12:16:18 PM UTC+1, Geert-Jan Brits wrote:

I'm banging my head against the wall on trying on faceting on
nested-documents with:

a filter on the nested documents

a filter on the 'parent' documents

To me the query (at end of post) should just work. Explanation:

a queryfilter is created limiting all hits AND facets to only match
against parent-docs with field rec=80

a queryfilter is created limiting all hits AND facets to only match
against parent-docs which have a nested-doc with ddp.ddpindex = "b" (this
filter is defined as scope: 'scope_ddp_exists')

Then I'd like to facet on nested field 'ddp.price' using just defined
scope 'scope_ddp_exists'.
To my understanding (since facets should already be contrained by
'rec=80') this should return nested docs:

of which the parent is contrained to rec=80

of which the nested-doc itself has the field ddpindex='b'

However, the filter " rec=80" isn't taken into account at all.

What's wrong with this filter? It is as if 'scope_ddp_exists' is placed
directly under the global-scope, instead of taking queryfilters set under
scope 'main' into account?
If this is intended, is there any other way to construct the query to
facet on scope 'scope_ddp_exists' while still taking the other
(query)-filters into account?

{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"and": [
{
"term": {"rec": 80}
},
{
"nested":{
"_scope": "scope_ddp_exists",
"path":"ddp",
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"term":{"ddp.ddpindex":"b"}
}
}
}
}
}
]
}
}
},
"facets": {
"priceslider": {
"terms": {
"field": "ddp.price"
},
"scope": "scope_ddp_exists"
}
}
}

Thanks,
Geert-Jan

--

Geert_Jan_Brits · December 21, 2012, 11:31am

Sorry about still confusing the nomenclature. When talking about the
'parent'-doc I mean the document on top-level which contains the
nested-documents. I'm familiar that this is different than parent-child
relationships

On Friday, December 21, 2012 12:17:27 PM UTC+1, Geert-Jan Brits wrote:

On Friday, December 21, 2012 12:16:18 PM UTC+1, Geert-Jan Brits wrote:

I'm banging my head against the wall on trying on faceting on
nested-documents with:

a filter on the nested documents

a filter on the 'parent' documents

To me the query (at end of post) should just work. Explanation:

a queryfilter is created limiting all hits AND facets to only match
against parent-docs with field rec=80

a queryfilter is created limiting all hits AND facets to only match
against parent-docs which have a nested-doc with ddp.ddpindex = "b" (this
filter is defined as scope: 'scope_ddp_exists')

Then I'd like to facet on nested field 'ddp.price' using just defined
scope 'scope_ddp_exists'.
To my understanding (since facets should already be contrained by
'rec=80') this should return nested docs:

of which the parent is contrained to rec=80

of which the nested-doc itself has the field ddpindex='b'

However, the filter " rec=80" isn't taken into account at all.

What's wrong with this filter? It is as if 'scope_ddp_exists' is placed
directly under the global-scope, instead of taking queryfilters set under
scope 'main' into account?
If this is intended, is there any other way to construct the query to
facet on scope 'scope_ddp_exists' while still taking the other
(query)-filters into account?

{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"and": [
{
"term": {"rec": 80}
},
{
"nested":{
"_scope": "scope_ddp_exists",
"path":"ddp",
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"term":{"ddp.ddpindex":"b"}
}
}
}
}
}
]
}
}
},
"facets": {
"priceslider": {
"terms": {
"field": "ddp.price"
},
"scope": "scope_ddp_exists"
}
}
}

Thanks,
Geert-Jan

--

Igor_Motov · December 21, 2012, 12:27pm

It's a known
issue Inconsistent results of facets running with a nested query scope · Issue #2398 · elastic/elasticsearch · GitHub

On Friday, December 21, 2012 6:31:08 AM UTC-5, Geert-Jan Brits wrote:

Sorry about still confusing the nomenclature. When talking about the
'parent'-doc I mean the document on top-level which contains the
nested-documents. I'm familiar that this is different than parent-child
relationships

On Friday, December 21, 2012 12:17:27 PM UTC+1, Geert-Jan Brits wrote:

On Friday, December 21, 2012 12:16:18 PM UTC+1, Geert-Jan Brits wrote:

I'm banging my head against the wall on trying on faceting on
nested-documents with:

a filter on the nested documents

a filter on the 'parent' documents

To me the query (at end of post) should just work. Explanation:

a queryfilter is created limiting all hits AND facets to only match
against parent-docs with field rec=80

a queryfilter is created limiting all hits AND facets to only match
against parent-docs which have a nested-doc with ddp.ddpindex = "b" (this
filter is defined as scope: 'scope_ddp_exists')

Then I'd like to facet on nested field 'ddp.price' using just defined
scope 'scope_ddp_exists'.
To my understanding (since facets should already be contrained by
'rec=80') this should return nested docs:

of which the parent is contrained to rec=80

of which the nested-doc itself has the field ddpindex='b'

However, the filter " rec=80" isn't taken into account at all.

What's wrong with this filter? It is as if 'scope_ddp_exists' is placed
directly under the global-scope, instead of taking queryfilters set under
scope 'main' into account?
If this is intended, is there any other way to construct the query to
facet on scope 'scope_ddp_exists' while still taking the other
(query)-filters into account?

{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"and": [
{
"term": {"rec": 80}
},
{
"nested":{
"_scope": "scope_ddp_exists",
"path":"ddp",
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"term":{"ddp.ddpindex":"b"}
}
}
}
}
}
]
}
}
},
"facets": {
"priceslider": {
"terms": {
"field": "ddp.price"
},
"scope": "scope_ddp_exists"
}
}
}

Thanks,
Geert-Jan

--

Geert_Jan_Brits · December 21, 2012, 12:44pm

Ahh somehow glad I'm not going insane. Something that would trivially solve
my use-case (and would be of much use in general imo) , thinking out loud
here, would be to be able to set '_scope' on 'container-filters' such and
'bool' as well as 'and' as long as that container-filter contains a
nested-filter as one of it's elements.
This would allow for creating arbitrary complex filters when faceting on
nested documents.

2012/12/21 Igor Motov imotov@gmail.com

It's a known issue
Inconsistent results of facets running with a nested query scope · Issue #2398 · elastic/elasticsearch · GitHub

On Friday, December 21, 2012 6:31:08 AM UTC-5, Geert-Jan Brits wrote:

Sorry about still confusing the nomenclature. When talking about the
'parent'-doc I mean the document on top-level which contains the
nested-documents. I'm familiar that this is different than parent-child
relationships

On Friday, December 21, 2012 12:17:27 PM UTC+1, Geert-Jan Brits wrote:

On Friday, December 21, 2012 12:16:18 PM UTC+1, Geert-Jan Brits wrote:

I'm banging my head against the wall on trying on faceting on
nested-documents with:

a filter on the nested documents

a filter on the 'parent' documents

To me the query (at end of post) should just work. Explanation:

a queryfilter is created limiting all hits AND facets to only match
against parent-docs with field rec=80

a queryfilter is created limiting all hits AND facets to only match
against parent-docs which have a nested-doc with ddp.ddpindex = "b" (this
filter is defined as scope: 'scope_ddp_exists')

Then I'd like to facet on nested field 'ddp.price' using just defined
scope 'scope_ddp_exists'.
To my understanding (since facets should already be contrained by
'rec=80') this should return nested docs:

of which the parent is contrained to rec=80

of which the nested-doc itself has the field ddpindex='b'

However, the filter " rec=80" isn't taken into account at all.

What's wrong with this filter? It is as if 'scope_ddp_exists' is
placed directly under the global-scope, instead of taking queryfilters set
under scope 'main' into account?
If this is intended, is there any other way to construct the query to
facet on scope 'scope_ddp_exists' while still taking the other
(query)-filters into account?

{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"and": [
{
"term": {"rec": 80}
},
{
"nested":{
"_scope": "scope_ddp_exists",
"path":"ddp",
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"term":{"ddp.ddpindex":"b"}
}
}
}
}
}
]
}
}
},
"facets": {
"priceslider": {
"terms": {
"field": "ddp.price"
},
"scope": "scope_ddp_exists"
}
}
}

Thanks,
Geert-Jan

--

--

Geert_Jan_Brits · December 21, 2012, 12:58pm

Another approach would be

github.com/elastic/elasticsearch

Expose matching nested docs to the query and sort

opened 10:43AM - 08 Oct 11 UTC

closed 03:06PM - 10 May 13 UTC

clintongormley

Hiya I have a use case for nested docs which, currently, can not be supported e…asily. For instance, have a look at this doc: ``` { "location" : { "lat" : "8.31667", "lon" : "52.38333" }, "parent_ids" : [ "2862926", "2921044" ], "contexts" : [ { "context" : "/en", "rank" : 2, "tokens" : [ { "tokens" : [ "steinbrink" ] }, { "tokens" : [ "steinbrink", "germany" ] } ], "label" : { "short" : "Steinbrink", "long" : "Steinbrink, Germany" } }, { "context" : "/de", "rank" : 3, "tokens" : [ { "tokens" : [ "steinbrink" ] }, { "tokens" : [ "steinbrink", "deutschland" ] } ], "label" : { "short" : "Steinbrink", "long" : "Steinbrink, Deutschland" } } ] } ``` `contexts` and `tokens` are both nested docs. I would like to run query like: - filter by `contexts.context == '/de'` - sort by: - `contexts.context.rank desc` - `contexts.label.short asc` but of course I have no way of pulling out the single matching context and using the values in that sub-doc. I could use a script sort like the horribly inefficient: ``` { "sort" : [ { "params" : { "name" : "/de" }, "_script" : { "script" : "for (context: contexts) { if (context.context == name) { return context.rank }}", "order" : "desc", "type" : "number" } }, { "_script" : { "params" : { "name" : "/de" }, "script" : "for (context: contexts) { if (context.context == name) { return context.label.short }}", "order" : "asc", "type" : "string" } } ] } ``` Or I may want to do: - filter by `context=='/de'` - find doc with tokens that best match `stein*` - return the `tokens` that match best as a field Or have a slightly different doc structure: ``` "tokens" : [ { "label": "Steinbrink", "tokens" : [ "steinbrink" ] }, { "label": "Steinbrink, Germany", "tokens" : [ "steinbrink, germany" ] }, ], ``` and return the `label` associated with the `tokens` that best match. Even with a script, I couldn't find the best matching `tokens`. In this example, I only have two contexts, and so could have changed the doc structure to something like: ``` contexts: { en: {....}, de: {....} } ``` But I have other use cases where I have hundreds of contexts, which would result in thousands of fields. Would it be possible to add functionality to: - expose the best matching child (or children, with the best matching first in an array) in a `_scope` so we could do something like: "sort": [ { "_child.myscope[0].rank": "desc" }, { "_child.myscope[0].label.short": "asc"} ], "fields": [ "_child.otherscope[0],tokens.label" ] - return the scores for matching children in a `_scope` eg: { "children": { "myscope": [5,3,0,2,0], "otherscope": [0,1,0] } } Not sure how feasible this is, and if it is feasible, there is probably a much more elegant way of doing it. thanks clint

and return the matched nested-document and 'merge' it in the doc at
top-level.
Afterwards facet on the fields of the merged nested doc as if they are
properties of the parent.

Incredibly powerful, since sorting, filtering, faceting, returning fields,
could just be done without the client even caring if the fields are from a
matched nested doc or from the main doc itself..
Oh well, pie in the skie at this moment I guess.

2012/12/21 Geert-Jan Brits gbrits@gmail.com

Ahh somehow glad I'm not going insane. Something that would trivially
solve my use-case (and would be of much use in general imo) , thinking out
loud here, would be to be able to set '_scope' on 'container-filters' such
and 'bool' as well as 'and' as long as that container-filter contains a
nested-filter as one of it's elements.
This would allow for creating arbitrary complex filters when faceting on
nested documents.

2012/12/21 Igor Motov imotov@gmail.com

It's a known issue
Inconsistent results of facets running with a nested query scope · Issue #2398 · elastic/elasticsearch · GitHub

On Friday, December 21, 2012 6:31:08 AM UTC-5, Geert-Jan Brits wrote:

Sorry about still confusing the nomenclature. When talking about the
'parent'-doc I mean the document on top-level which contains the
nested-documents. I'm familiar that this is different than parent-child
relationships

On Friday, December 21, 2012 12:17:27 PM UTC+1, Geert-Jan Brits wrote:

On Friday, December 21, 2012 12:16:18 PM UTC+1, Geert-Jan Brits wrote:

I'm banging my head against the wall on trying on faceting on
nested-documents with:

a filter on the nested documents

a filter on the 'parent' documents

To me the query (at end of post) should just work. Explanation:

a queryfilter is created limiting all hits AND facets to only match
against parent-docs with field rec=80

a queryfilter is created limiting all hits AND facets to only match
against parent-docs which have a nested-doc with ddp.ddpindex = "b" (this
filter is defined as scope: 'scope_ddp_exists')

Then I'd like to facet on nested field 'ddp.price' using just defined
scope 'scope_ddp_exists'.
To my understanding (since facets should already be contrained by
'rec=80') this should return nested docs:

of which the parent is contrained to rec=80

of which the nested-doc itself has the field ddpindex='b'

However, the filter " rec=80" isn't taken into account at all.

What's wrong with this filter? It is as if 'scope_ddp_exists' is
placed directly under the global-scope, instead of taking queryfilters set
under scope 'main' into account?
If this is intended, is there any other way to construct the query to
facet on scope 'scope_ddp_exists' while still taking the other
(query)-filters into account?

{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"and": [
{
"term": {"rec": 80}
},
{
"nested":{
"_scope": "scope_ddp_exists",
"path":"ddp",
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"term":{"ddp.ddpindex":"b"}
}
}
}
}
}
]
}
}
},
"facets": {
"priceslider": {
"terms": {
"field": "ddp.price"
},
"scope": "scope_ddp_exists"
}
}
}

Thanks,
Geert-Jan

--

--

Topic		Replies	Views
Facet filters on parent fields Elasticsearch	1	354	July 6, 2017
Parent conditions in NESTED or SCOPED facets Elasticsearch	3	375	July 6, 2017
Facet on Nested Objects filtered by Outer object? Elasticsearch	10	1155	July 6, 2017
Facets on child documents Elasticsearch	7	592	July 6, 2017
Nested facet filters Elasticsearch	1	308	July 6, 2017

BUG?: faceting on nested-documents (with scope) doesn't take filter on parent-docs into account

Related topics