I'm using ES 5.6. It seems that the aggregations apply before the rescore is applied to the query. I say this because if you try to sort the buckets by a max "_score" it uses the score from the main query and not the rescore. More detail can be found here https://stackoverflow.com/questions/52212625/how-do-i-select-the-top-term-buckets-based-on-a-rescore-function-in-elasticsearc
Is there any way to perform the aggregations after the rescore? If not, how feasible would it be to write a plugin that performs the rescore function over the main query results, on each shard involved, then aggregate post rescore on each shard, then aggregate those aggregations on the query node of a cluster? If that's feasibly and the only way to achieve the result that I want, could anyone point me in the direction of an existing plugin that does something like this as a base to start from?