Multi-Leveled Nested Object Facet Search Requiring multiple Levels of Nested Object Access Does not access all fields


(Jessica) #1

I have a very complicated process I'm trying to complete which, to get the
data I need requires the following object format:

Document with values, one value is a nested document. That nested document
itself has a nested document:
general example:
Object do
Keywords do nested object
Name => 'string'
Records do nested object
RecName => 'string'
RecCount => 'integer'
end
end
end

I need to create a terms_stats facet which will facet on the Keywords.name
value, and will sum the Keywords.Records.RecCount values

I have an example for the documents and my queries/facet formatting at the
following gist: https://gist.github.com/anonymous/6457316

In this gist, the last two queries show:

  1. If I use nested:"keywords" then I can correctly facet on the
    Keywords.Name value,
  2. However it won't let me access the RecCount value and changing the
    nested to keywords.records gives no facet results.
    *I've also tried to have multiple nested values with both keywords and
    keywords.name, but it seems to only take whichever nested value I give last.

I really need to be able to access both levels of the documents in the same
facet. I need to have by Name, the total RecCounts and my documents need to
be set up this way for other users.

Please Help!

--J

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jessica) #2

Answer:

add :include_in_parent => 'true' for the sub-object, which does not have to
be nested and by doing nested : Keywords, all levels will be included.

On Thursday, September 5, 2013 6:01:45 PM UTC-5, Jessica wrote:

I have a very complicated process I'm trying to complete which, to get the
data I need requires the following object format:

Document with values, one value is a nested document. That nested document
itself has a nested document:
general example:
Object do
Keywords do nested object
Name => 'string'
Records do nested object
RecName => 'string'
RecCount => 'integer'
end
end
end

I need to create a terms_stats facet which will facet on the Keywords.name
value, and will sum the Keywords.Records.RecCount values

I have an example for the documents and my queries/facet formatting at the
following gist: https://gist.github.com/anonymous/6457316

In this gist, the last two queries show:

  1. If I use nested:"keywords" then I can correctly facet on the
    Keywords.Name value,
  2. However it won't let me access the RecCount value and changing the
    nested to keywords.records gives no facet results.
    *I've also tried to have multiple nested values with both keywords and
    keywords.name, but it seems to only take whichever nested value I give
    last.

I really need to be able to access both levels of the documents in the
same facet. I need to have by Name, the total RecCounts and my documents
need to be set up this way for other users.

Please Help!

--J

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Zachary Tong) #3

Gah, sorry, I had a reply written up to this but apparently never hit
submit. You're solution is correct and is really the only way to do it
without restructuring your data.

Below is the response that I never posted, provides some extra background:

I'm afraid you won't be able to do it, with this current layout. It's a

fundamental limitation to how nested documents are managed under the
surface.

Nested docs are basically "hidden" documents that are physically located
right next to the parent. They have their own internal Lucene ID and
behave just like documents. When you are doing queries or facets,
documents are handled one at a time. Think of it like a stream of
documents and you have a window of size 1. What this means is that you can
only "see" the data within that window, which is either the parent, the
child or the grand-child. You're window never holds two documents at the
same time, so you can never cross-relate data between parent and child.

Since you can only look at one document at a time, you need all the data
that you want to facet on that doc. One way to do this is the
include_in_parent option. This takes the value from the children and
de-normalizes them into the parent. It's important to note that this means
all "relational" data about the field is gone - it is simply a bag of
values that exists in the parent now (just like an "inner object"). If you
can tolerate that caveat, this will work great for you.

-Zach

On Friday, September 6, 2013 4:12:51 PM UTC-4, J wrote:

Answer:

add :include_in_parent => 'true' for the sub-object, which does not have
to be nested and by doing nested : Keywords, all levels will be included.

On Thursday, September 5, 2013 6:01:45 PM UTC-5, Jessica wrote:

I have a very complicated process I'm trying to complete which, to get
the data I need requires the following object format:

Document with values, one value is a nested document. That nested
document itself has a nested document:
general example:
Object do
Keywords do nested object
Name => 'string'
Records do nested object
RecName => 'string'
RecCount => 'integer'
end
end
end

I need to create a terms_stats facet which will facet on the
Keywords.name value, and will sum the Keywords.Records.RecCount values

I have an example for the documents and my queries/facet formatting at
the following gist: https://gist.github.com/anonymous/6457316

In this gist, the last two queries show:

  1. If I use nested:"keywords" then I can correctly facet on the
    Keywords.Name value,
  2. However it won't let me access the RecCount value and changing the
    nested to keywords.records gives no facet results.
    *I've also tried to have multiple nested values with both keywords and
    keywords.name, but it seems to only take whichever nested value I give
    last.

I really need to be able to access both levels of the documents in the
same facet. I need to have by Name, the total RecCounts and my documents
need to be set up this way for other users.

Please Help!

--J

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #4