# Missing documents when using partitions on a term aggregation

**URL:** <https://discuss.elastic.co/t/missing-documents-when-using-partitions-on-a-term-aggregation/237483>\
**Category:** Elasticsearch\
**Created:** [June 17, 2020, 2:07pm UTC](https://discuss.elastic.co/t/missing-documents-when-using-partitions-on-a-term-aggregation/237483 "2020-06-17T14:07:11Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jsvac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvac/32/47016_2.png) [@jsvac](https://discuss.elastic.co/u/jsvac)\
**Post date:** [June 17, 2020, 2:07pm UTC](https://discuss.elastic.co/t/missing-documents-when-using-partitions-on-a-term-aggregation/237483/1 "2020-06-17T14:07:11Z")

</div>

Hi all,

I have a relatively small index (about 5M docs) and each document has an organization\_id field. I need to list all the unique values for this field and I've implemented this using partitions by following this:

[https://www.elastic.co/guide/en/elasticsearch/reference/master/search-aggregations-bucket-terms-aggregation.html#\_filtering\_values\_with\_partitions](https://www.elastic.co/guide/en/elasticsearch/reference/master/search-aggregations-bucket-terms-aggregation.html#_filtering_values_with_partitions)

It works in most cases.. but I managed to find a few cases where the number of results returned does not sum up to the same amount when changing the number of partitions.

I know my query should return about 19000 values. If I do a single query with only 1 partition or with 5 partitions, I am getting more results then if I do 20 queries with 20 partitions. The missing docs (484 of them) are consistent as they are always the same.

I've double-checked to make sure I was not skipping or missing a partition so I am at loss to explain this behaviour. I am using AWS ElasticSearch 6.2.

any help/comment is welcome

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [June 18, 2020, 7:29am UTC](https://discuss.elastic.co/t/missing-documents-when-using-partitions-on-a-term-aggregation/237483/2 "2020-06-18T07:29:23Z")

</div>

Did you check the doc count error bounds returned in results was zero? A partition size that is too small can have inaccuracies which are reported.

If you’re not looking for any particular sort order to the results the ‘composite’ aggregation may be a simpler way to walk through the list.

---

<div class="post-metadata">

**Author:** ![jsvac](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvac/32/47016_2.png) [@jsvac](https://discuss.elastic.co/u/jsvac)\
**Post date:** [June 18, 2020, 11:25am UTC](https://discuss.elastic.co/t/missing-documents-when-using-partitions-on-a-term-aggregation/237483/3 "2020-06-18T11:25:07Z")

</div>

I will check for the error count to be sure but I believe they were all at zero.

I will have a look at the composite aggregation.

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 16, 2020, 11:25am UTC](https://discuss.elastic.co/t/missing-documents-when-using-partitions-on-a-term-aggregation/237483/4 "2020-07-16T11:25:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.