# How to determine the correct size for terms aggregation, which will produce accurate aggregation results?

**URL:** https://discuss.elastic.co/t/how-to-determine-the-correct-size-for-terms-aggregation-which-will-produce-accurate-aggregation-results/294771
**Category:** Elasticsearch
**Created:** [January 19, 2022, 6:44am UTC](https://discuss.elastic.co/t/how-to-determine-the-correct-size-for-terms-aggregation-which-will-produce-accurate-aggregation-results/294771 "2022-01-19T06:44:16Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Nishikant\_Tayade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nishikant_tayade/32/84159_2.png) [@Nishikant\_Tayade](https://discuss.elastic.co/u/Nishikant_Tayade)
#### Post date: [January 19, 2022, 6:44am UTC](https://discuss.elastic.co/t/how-to-determine-the-correct-size-for-terms-aggregation-which-will-produce-accurate-aggregation-results/294771/1 "2022-01-19T06:44:16Z")

</div>

As I read through document for Terms Aggregation, I came across the fact that the results from Term Aggregation are not always accurate, but we can increase the **size** to get the accurate results.

I know : -

1. How Query-Then-Fetch works.
2. How top terms are calculated at each shard(shard\_size) and then merge at co-ordinator node(size).
3. What "doc\_count\_error\_upper\_bound" means, and how it can help in determining that there may be error in top results and we need to increase the size.

But is there any mathematical approach or any other way, with help of which we can determine the correct size that we should ask for once we get in-accurate results for the first time?

---

<div class="post-metadata">

### Author: ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)
#### Post date: [January 19, 2022, 7:42am UTC](https://discuss.elastic.co/t/how-to-determine-the-correct-size-for-terms-aggregation-which-will-produce-accurate-aggregation-results/294771/2 "2022-01-19T07:42:31Z")

</div>

Maybe no,

Suppose you have 3 primary shards and `size=10`, `shard_size=25`, and the 10th term 'A' in the result were contained only in the result of 2 primary shards.  
If the count of the 25th term in the shard not returning 'A' is N, the count of 'A' in that shard should be any value between 0 and N. Then `doc_count_error_upper_bound` is N (or N-1, I'm not sure). `doc_count_error_upper_bound` is the sum of threasholds of shards not returning the bucket of that specific term.

About the rank of 'A' in such shards, there is no information contained in the response. Therefore, there is no way to calculate the necessary `size` or `shard_size` for accurate result. The only way to guarantee zero error should be use `shard_size` more than the terms cardinality.

---

<div class="post-metadata">

### Author: ![Nishikant\_Tayade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nishikant_tayade/32/84159_2.png) [@Nishikant\_Tayade](https://discuss.elastic.co/u/Nishikant_Tayade)
#### Post date: [January 19, 2022, 8:23am UTC](https://discuss.elastic.co/t/how-to-determine-the-correct-size-for-terms-aggregation-which-will-produce-accurate-aggregation-results/294771/3 "2022-01-19T08:23:59Z")

</div>

@Tomo_M Thanks for replying!!

> The only way to guarantee zero error should be use `shard_size` more than the terms cardinality.

Here what do you mean by terms cardinality ? Is it the count of unique terms, or count of all the terms?

---

<div class="post-metadata">

### Author: ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)
#### Post date: [January 19, 2022, 8:27am UTC](https://discuss.elastic.co/t/how-to-determine-the-correct-size-for-terms-aggregation-which-will-produce-accurate-aggregation-results/294771/4 "2022-01-19T08:27:13Z")

</div>

I meant the count of unique terms.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 16, 2022, 8:27am UTC](https://discuss.elastic.co/t/how-to-determine-the-correct-size-for-terms-aggregation-which-will-produce-accurate-aggregation-results/294771/5 "2022-02-16T08:27:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
