Hello,
I would appreciate any help/insight that can be provided on the below issues.
I was trying to implement Composite Aggregations with the 6.3 Rest Client and noticed that it didn't seem to be supported yet. Am I correct? And are there any examples of using the Composite Aggregation with the Low Level Client?
I would've preferred to use Composite, as opposed to Terms, Aggregation so I don't have to set a large (100,000+) size parameter to get back the necessary buckets (I was planning on using the 'after' feature of Composite Aggregation to scroll through and collect the results). But, it seems I might have to go with Terms.
With Terms Aggregation, I saw the partitioning feature - although it isn't the exact same as pagination, it seems like it can fulfill my needs. I'd run multiple partitioned aggregations and collect my aggregations client-side. They wont be in a correct sorted order that I would've achieved with a Composite Aggregation or a single non-partitioned Term Aggregation, right?
And at what point is partitioning necessary?
Should I only do this if I get memory/cpu errors without partitioning and not pre-optimize? I am not returning any hits - I am only performing aggregations on the narrowed down data (via bool filter queries).
If I don't partition the aggregations, I would set a large 100,000 size parameter and there will be another nested term aggregation with the same size of 100,000 for each term (I can't partition this nested Term Aggregation to my understanding, and 100,000 is the high end of the number of terms here - although it'll likely be like 5 or 6 normally).
Is ES making large 100,000 size internal data structures that may not be filled even close to their capacity? I was reading that there inefficenies of setting arbitrarily large size paramters has decreased in later versions of ES.
Thank you!