# Elasticsearch terms aggregation with partition does not retuning equal bucket **URL:** \ **Category:** Elasticsearch\ **Created:** [March 10, 2022, 2:58pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346 "2022-03-10T14:58:59Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![truptir](https://avatars.discourse-cdn.com/v4/letter/t/58f4c7/32.png) [@truptir](https://discuss.elastic.co/u/truptir)\ **Post date:** [March 10, 2022, 2:58pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346/1 "2022-03-10T14:58:59Z")
I have an ES index and I fetch data using term aggregation and partition. I am using the cardinality aggregation to get the total number of buckets and calculating num\_partitions, using default size values 10. E.g. Cardinality aggregation returns the count 3655. partition size = 3655/10 = 366 When I execute term aggregation with num\_partitions = 366, size = 10 it returns me the following number of buckets in the requests = 6, 2, 10, 8 10, 10, 9,... 10 the sum of all buckets is less than 3665 and even after the last page, it skips many data. I would expect it to go like this 10,10,10,10,.....5 So, the question is - - Why each partition does not include 10 buckets as per size? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [March 10, 2022, 10:44pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346/2 "2022-03-10T22:44:51Z")
> [@truptir](#): > > Why each partition does not include 10 buckets as per size? This slight unevenness is to be expected given the use of the widely used [hash partitioning](https://www.techopedia.com/definition/31996/hash-partitioning) technique. If the results across all partitions don’t add up then that would be a bug - but I doubt that is the case. You may have overlooked an error bounds that was reported with some of the terms agg results. ---
**Author:** ![truptir](https://avatars.discourse-cdn.com/v4/letter/t/58f4c7/32.png) [@truptir](https://discuss.elastic.co/u/truptir)\ **Post date:** [March 11, 2022, 2:07pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346/3 "2022-03-11T14:07:29Z")
> [@Mark\_Harwood](#): > > If the results across all partitions don’t add up then that would be a bug - but I doubt that is the case. Hi @Mark_Harwood, I think this is a bug. For Every index, I m facing the same issue. Below are some more cases that return a different number of buckets for the same index. Cardinality aggregation returns count: 500 _ **Case 1** _ size: 500 num\_partitions: 1 return total buckets: 430 _ **Case 2** _ size: 250 num\_partitions: 2 return total buckets: 217 + 213 = 430 _ **Case 3** _ size: 100 num\_partitions: 5 return total buckets: 72+79+80+100+94 = 425 _ **Case 4** _ size: 50 num\_partitions: 10 return total buckets: 37+34+39+50+48+35+45+41+48+46 = 423 And as I increase the number of partitions the sum of buckets across all partitions is very less. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [March 11, 2022, 2:41pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346/4 "2022-03-11T14:41:50Z")
Note in cases 3 and 4 there are partitions returning exactly the number of buckets requested in your ‘size’ parameter. That is a strong indication that there’s \> N buckets to be returned but you only asked to see N ---
**Author:** ![truptir](https://avatars.discourse-cdn.com/v4/letter/t/58f4c7/32.png) [@truptir](https://discuss.elastic.co/u/truptir)\ **Post date:** [March 14, 2022, 2:29pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346/5 "2022-03-14T14:29:54Z")
Okay, I understood. But First I calculated no. of partitions based on the total count and size so that it fits total data. Can you suggest then what should be the best way to calculate the no. of partitions so that it does not skip any bucket When I have total count and size. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [March 14, 2022, 8:03pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346/6 "2022-03-14T20:03:56Z")
Calculate the number of partitions so that you’re working with a manageable subset of the data for the analysis you want to do. Too small a partition number = many unique keys per partition = potential overload of memory or inaccuracies Too large a partition number = many client requests required = slow Whatever partition number you settle on, up the ‘size’ used in the request. E.g if you think the partition will produce roughly 1000 keys per partition set the size to 1500 to allow for the unevenness we talked about previously. Setting it to 1000 exactly would be a mistake for the reason I outlined in my last comment. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 11, 2022, 8:04pm UTC](https://discuss.elastic.co/t/elasticsearch-terms-aggregation-with-partition-does-not-retuning-equal-bucket/299346/7 "2022-04-11T20:04:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.