Variable Width Histogram not working with top hits subaggregation

Elastic Stack Elasticsearch
1

I'm trying to use variable width histogram to group results with similar relevance scores and then apply a secondary sort on them via a top hits aggregation. This is so that folks can sort on relevance and then also sort on a secondary field (today all the scores are too granular, so the secondary sort never comes into play).

Somehow however, when I do this, while the variable width aggregation creates nice bucket ranges for my scores, the top hits aggregation actually is returning results unrelated to the buckets on hand. It's almost as if they're randomly sorted into the wrong buckets. This exact same functionality works fine if the nested aggregation is a range aggregation or a normal histogram. Any idea what I might be doing wrong here? I'm using Elasticsearch 7.10.

"aggs": {
    "scores": {
    //   "histogram": { // This works fine if in place of the variable width
    //     "script": "_score",
    //     "interval": 2,
    //     "min_doc_count": 1
    //   },
    "variable_width_histogram": {
        "script": "_score",
        "buckets": 3
      },
      "aggs": {
        "top_hits_agg": {
          "top_hits": {
            "sort": [
              {
                 "numeric_kg_p_location*custom*70452*number*0": {
                   "order": "desc"
                 }
              }
            ],
            "_source": {
              "includes": [ 
                "_id", 
                "full_kg_p_location*business_name_en", 
                "numeric_kg_p_location*custom*70452*number*0"
                 ],
                 "excludes": ["vector*"]
            },
            "from": 0,
            "size": 5
          }
        }
      }
    }
  }

Thanks!