Count nested documents only?

[bkazez]

Hi, in my index, each document represents a Record Group, which includes an array of Records -
type: nested. I need the count of Records only, not Record Groups. I've seen several people looking for this kind of thing over time and wondering if we all missed a way to do this?

https://stackoverflow.com/questions/45143183/count-nested-documents-along-with-parent-documents-in-size-parameter-of-query
https://stackoverflow.com/questions/44576150/count-nested-documents-in-elasticsearch
https://stackoverflow.com/questions/45143183/count-nested-documents-along-with-parent-documents-in-size-parameter-of-query?rq=1

Get count of parent documents only

Hi all, I have a document which has some nested documents. When I do something like the following, the count that comes back includes the nested document: GET loandetails2/loandetail/_count { "query": { "bool": { "should": [ { "match": { "visibility": { "query": "3" } } } ] } } } The above query is just a simplified version of what I'm actually doing. Sometimes I'm really trying to get a count based on fields in the parent document, and fields in the nested documents. But I …

Thank you,
Ben

[val (Val Crettaz)]

Using nested inner_hits you can get the count of documents that are nested inside your top-level documents. However, you won't get a total count for the whole result set, you need to iterate over each hit and sum up the nested count on the client side.

[bkazez]

That's a bummer, since many records have just 1 or 2 nested documents, and the result sets can be very large (the eventual solution needs to work even for 15k results). It seems people have needed this for quite some time and I have to imagine that internally, since the nested objects are separate documents, it must be possible to get this count faster from elasticsearch's internals than from a client-side count? Any chance of seeing an API for this someday?

[oguz]

I also wanted to get inner hit count before but since it is not possible i changed my mapping to a parent-child mapping. This way you can get parent-child objects separately with has_child, has_parent queries.

[bkazez]

Thanks @oguz - I considered parent-child queries, but the Elasticsearch guide says that they 'can be 5 to 10 times slower than the equivalent nested query'. Unfortunately query speed is most important for this application. Any other thoughts?

[dadoonet (David Pilato)]

Can't you compute this value at index time?

[bkazez]

Ah of course. I did that and now I have the total result count but it doesn't work with "size" for result pagination (Pagination in aggregation). I can add a filter aggregation, but I believe I can't set the size on that filter. I'm an ES beginner, though...hopefully I'm missing something!

[Magnus_Kessler]

Have a look if your use case allows you to use the approach described here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_filtering_values_with_partitions

With ES 6.1, there's also new functionality specifically designed to allow deep pagination (similar to the scroll API) in aggregations:
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html