I am trying to implement the inbuilt pagination feature available in elastic search composite aggregation on aggregated data. This works perfectly fine until we are not removing certain buckets based on conditions of aggregated fields. To remove buckets I am using BucketSelectorAggregation.
I understand that composite aggregation doesn't support pipeline aggregation fully and the reason for the issue is after_key and size always consider the aggregation response for pagination, rather than filtred aggregated data.
If anybody can help on this to apply the pagination on the aggregated filtered bucket using some kind of filter or other option without using pipeline aggregation that would be great.
Example :
id sum
1 100
2 200
3 400
4 500
5 100
Query after aggregation to get 2 records
Result
id sum
1 100
2 200
Query after aggregation and BucketSelectorAggregation to get 2 records where sum value 100
Result
1 100
Expected Result
1 100
5 100