Count nested documents only?

Elastic Stack Elasticsearch
1

Hi, in my index, each document represents a Record Group, which includes an array of Records -
type: nested. I need the count of Records only, not Record Groups. I've seen several people looking for this kind of thing over time and wondering if we all missed a way to do this?

https://stackoverflow.com/questions/45143183/count-nested-documents-along-with-parent-documents-in-size-parameter-of-query
https://stackoverflow.com/questions/44576150/count-nested-documents-in-elasticsearch
https://stackoverflow.com/questions/45143183/count-nested-documents-along-with-parent-documents-in-size-parameter-of-query?rq=1

Thank you,
Ben

2

Using nested inner_hits you can get the count of documents that are nested inside your top-level documents. However, you won't get a total count for the whole result set, you need to iterate over each hit and sum up the nested count on the client side.

3

That's a bummer, since many records have just 1 or 2 nested documents, and the result sets can be very large (the eventual solution needs to work even for 15k results). It seems people have needed this for quite some time and I have to imagine that internally, since the nested objects are separate documents, it must be possible to get this count faster from elasticsearch's internals than from a client-side count? Any chance of seeing an API for this someday?

4

I also wanted to get inner hit count before but since it is not possible i changed my mapping to a parent-child mapping. This way you can get parent-child objects separately with has_child, has_parent queries.

5

Thanks @oguz - I considered parent-child queries, but the Elasticsearch guide says that they 'can be 5 to 10 times slower than the equivalent nested query'. Unfortunately query speed is most important for this application. Any other thoughts?

6

Can't you compute this value at index time?

7

Ah of course. I did that and now I have the total result count but it doesn't work with "size" for result pagination (Pagination in aggregation). I can add a filter aggregation, but I believe I can't set the size on that filter. I'm an ES beginner, though...hopefully I'm missing something!

8

Have a look if your use case allows you to use the approach described here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_filtering_values_with_partitions

With ES 6.1, there's also new functionality specifically designed to allow deep pagination (similar to the scroll API) in aggregations:
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.