Java REST API - No way to perform filters on aggregations?

[rarestg]

I'm trying to do two things:

• Filter results with a geo-bounding box
• Filter results within a timeframe
• Query for an aggregation, the average of a certain data field

I've been able to do this with the following POST:

  curl -XGET "http://localhost:9200/data/_search" -H 'Content-Type: application/json' -d'
  {
    "size": 0,
    "_source": {
      "excludes": []
    },
    "aggs": {
      "filter_agg": {
        "filter": {
          "geo_bounding_box": {
            "place_lat": {
              "top_left": {
                "lat": 37.47
                "lon": -122.41
              },
              "bottom_right": {
                "lat": 37.27,
                "lon": -121.72
              }
            }
          }
        },
        "aggs": {
          "2": {
            "geohash_grid": {
              "field": "place_lat",
              "precision": 6
            },
            "aggs": {
              "1": {
                "avg": {
                  "field": "coordinates"
                }
              },
              "3": {
                "geo_centroid": {
                  "field": "coordinates"
                }
              }
            }
          }
        }
      }
    },
    "stored_fields": [
      "*"
    ],
    "script_fields": {},
    "docvalue_fields": [
      "begin_time",
      "dropoff_time",
      "request_time"
    ],
    "query": {
      "bool": {
        "must": [
          {
            "range": {
              "request_time": {
                "gte": 1530427399999,
                "lte": 1530428399999,
                "format": "epoch_millis"
              }
            }
          }
        ],
        "filter": [
          {
            "match_all": {}
          }
        ],
        "should": [],
        "must_not": []
      }
    }
  }'

But I can't seem to do this using the Java High-Level REST Api.

I can create a geohashGrid aggregation, and sub-aggregation based on whatever field I'm interested in. However I'm not sure how to add the two filters. The following hangs on client.search():

    // Build the bounding box filter
    GeoBoundingBoxQueryBuilder queryBuilder = QueryBuilders
            .geoBoundingBoxQuery("bounding_box")
            .setCorners(topLeft, bottomRight);

    // Aggregate on a field
    GeoGridAggregationBuilder aggregation = AggregationBuilders
            .geohashGrid("by_region")
            .field("coordinates")
            .precision(6);
    aggregation.subAggregation(AggregationBuilders
            .avg("my_data")
            .field("my_data"));

Help is absolutely appreciated. Thank you!