Aggregation Partition Number

Elastic Stack Elasticsearch
1

I have some crores data in elastic search. Now, i am trying to Fetch data with Some Aggregation and sub aggregations. (total 15 aggregations including 14 sub aggregations).

For those data, I come to know that elastic search has limit 10000 buckets in a single response.

I am thinking about to use Partition in term aggragation. (please refer Link)

Can someone explain how I can decide Total number of Partition for those records?

2

If you are happy with sorting term buckets by their key Iā€™d suggest using the ā€˜compositeā€™ aggregation and its ā€˜afterā€™ parameter.

The reference docs for the terms aggregation give some advice on picking partition numbers.

3

Thanks @Mark_Harwood !!

I used same thing in other place.
but i need to Do Sum of a "Salary" column after these 14 or 15 Group by.
Can it possible in Composite Aggregation ? If Yes, Please suggest a way.

If it is not possible in Composite Query, then How can i achieve this scenario? Any thoughts ? Thank you in advanced.

4

You can nest arbitrary aggregations using composite - you just canā€™t sort the top-level buckets in the tree by anything other than their key.

5

Thanks @Mark_Harwood for you suggestion and time. It really works.

1 Like
6

Admittedly there's a lot of different approaches with different trade-offs.
I tried to capture some of that decision-making in a wizard.

1 Like
7

Really Awesome !!! .... That Wizard will help and save time in many situation.

Youā€™re the best. I wouldnā€™t have finished this on time without you.

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.