Aggregation Partition Number

Elastic Stack Elasticsearch
1

I have some crores data in elastic search. Now, i am trying to Fetch data with Some Aggregation and sub aggregations. (total 15 aggregations including 14 sub aggregations).

For those data, I come to know that elastic search has limit 10000 buckets in a single response.

I am thinking about to use Partition in term aggragation. (please refer Link)

Can someone explain how I can decide Total number of Partition for those records?

2

If you are happy with sorting term buckets by their key I’d suggest using the ‘composite’ aggregation and its ‘after’ parameter.

The reference docs for the terms aggregation give some advice on picking partition numbers.

3

Thanks @Mark_Harwood !!

I used same thing in other place.
but i need to Do Sum of a "Salary" column after these 14 or 15 Group by.
Can it possible in Composite Aggregation ? If Yes, Please suggest a way.

If it is not possible in Composite Query, then How can i achieve this scenario? Any thoughts ? Thank you in advanced.

4

You can nest arbitrary aggregations using composite - you just can’t sort the top-level buckets in the tree by anything other than their key.

5

Thanks @Mark_Harwood for you suggestion and time. It really works.

1 Like
6

Admittedly there's a lot of different approaches with different trade-offs.
I tried to capture some of that decision-making in a wizard.

1 Like
7

Really Awesome !!! .... That Wizard will help and save time in many situation.

You’re the best. I wouldn’t have finished this on time without you.

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.