Facets and filtering on nested documents

Simon_Hutchinson · March 12, 2013, 8:12pm

Hi All,

I am looking for some advice on the best way to facet on multiple nested
objects.

The following gist contains some curl requests that will help to illustrate
what I am doing.

gist.github.com

https://gist.github.com/sihutch/5146380

es-nested-facets


curl -XDELETE 'http://localhost:9200/exp/'

#Create the index

curl -XPUT 'http://localhost:9200/exp/'

#Configure the mapping

curl -XPUT localhost:9200/exp/content/_mapping -d '

This file has been truncated. show original

In the final curl command in the above gist I am successfully able to get
facets from the two nested documents. The response can be seen here:

gist.github.com

https://gist.github.com/sihutch/5146432

es-nest-facets-response

"facets" : {
    "contributors.contributorRole.untouched" : {
      "_type" : "terms",
      "missing" : 0,
      "total" : 2,
      "other" : 0,
      "terms" : [ {
        "term" : "contributor role 2",
        "count" : 1
      }, {

This file has been truncated. show original

What I have not been able to work out (if it is possible at all) is how to
get facets from nested document "contributors" whilst filtering on the
other nested document "identifiers".

I would appreciate any advice anyone has to offer on:

a) Whether this is possible (From my limited understanding of nested
documents actually being indexed separately my guess is that it will not
work)
b) A different approach that might help me achieve the same goal i.e
faceted search on multiple nested documents without the problems of "cross
object" search

Best regards

Si

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Clinton_Gormley · March 13, 2013, 12:55pm

Hi Simon

I am looking for some advice on the best way to facet on multiple
nested objects.

The following gist contains some curl requests that will help to
illustrate what I am doing.

es-nested-facets · GitHub

In the final curl command in the above gist I am successfully able to
get facets from the two nested documents. The response can be seen
here: es-nest-facets-response · GitHub

What I have not been able to work out (if it is possible at all) is
how to get facets from nested document "contributors" whilst
filtering on the other nested document "identifiers".

I really want to help, but I don't quite follow what you want to do
Could you give some example output that you would like to see?

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Simon_Hutchinson · March 13, 2013, 1:56pm

Hi Clint,

Thanks for replying

I really want to help, but I don't quite follow what you want to do
Could you give some example output that you would like to see?

I shall try to clarify what I want.

If I have a document A that has two "inner objects" B and C each mapped as
nested is it possible to apply a facet filter for B to a facet of C?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Clinton_Gormley · March 14, 2013, 10:30am

Hiya

If I have a document A that has two "inner objects" B and C each
mapped as nested is it possible to apply a facet filter for B to a
facet of C?

OK - I think I've got it.

No - you can't filter on one nested document, and facet on the other.
They are separate documents, and they can't see each other's data.

What you can do (which may or may not suit your requirements) is to set
"include_in_parent" or "include_in_root" to true. This indexes the data
in the nested objects both as nested (ie independent) documents, AND in
the flattened form that you would have if you had specified type
"object" instead of type "nested".

so eg a document like:
{
one: [ {foo: 1}, {foo:2}],
two: [ {bar: 1}, {bar:2}]
}
where "one" and "two" are {type: "nested", include_in_parent: true}
would be indexed as:

parent doc:

{
one.foo: [1,2],
two.bar: [1,2]
}

plus nested docs:

{ "one.foo": 1}
{ "one.foo": 2}
{ "two.bar": 1}
{ "two.bar": 2}

So then you would be able to facet on values in the parent doc, instead
of the nested docs. However, on the parent doc, you don't have the
ability to correlate values within nested docs. Your filter can only
ask "(do any of the nested docs have value X) and (do any of the nested
docs have value Y)".

This may or may not suit your needs.

The only other way around it would be to repeat the data within each
nested doc.

clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Simon_Hutchinson · March 14, 2013, 11:03am

If I have a document A that has two "inner objects" B and C each
mapped as nested is it possible to apply a facet filter for B to a
facet of C?

OK - I think I've got it.

No - you can't filter on one nested document, and facet on the other.
They are separate documents, and they can't see each other's data.

What you can do (which may or may not suit your requirements) is to set
"include_in_parent" or "include_in_root" to true. This indexes the data
in the nested objects both as nested (ie independent) documents, AND in
the flattened form that you would have if you had specified type
"object" instead of type "nested".

so eg a document like:
{
one: [ {foo: 1}, {foo:2}],
two: [ {bar: 1}, {bar:2}]
}
where "one" and "two" are {type: "nested", include_in_parent: true}
would be indexed as:

parent doc:

{
one.foo: [1,2],
two.bar: [1,2]
}

plus nested docs:

{ "one.foo": 1}
{ "one.foo": 2}
{ "two.bar": 1}
{ "two.bar": 2}

So then you would be able to facet on values in the parent doc, instead
of the nested docs. However, on the parent doc, you don't have the
ability to correlate values within nested docs. Your filter can only
ask "(do any of the nested docs have value X) and (do any of the nested
docs have value Y)".

This may or may not suit your needs.

The only other way around it would be to repeat the data within each
nested doc.

The behaviour is as I expected but your clarification is much appreciated.
I have gone with include_in_parent: true for now. I will have to consider
the implications of "cross object" that results from this.

Tips hat graciously to Clint

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.