How about an ability to custom define 'the most relevant document', e.g. order the aggregated docs based on most recently updated rather than based on the count.
To illustrate by reusing example with the tags: order the tags not based on how many documents containing a certain tag, but order the tags based on the recency of the documents containing a certain tag.
Alternatively what if we could use nested aggregation with ability to include fields in the lowest resolution bucket. E.g. aggs -> terms by fieldA -> max by fieldB -> include fieldC in the output.