Bucket selector null pointer exception

[Roy_Levy]

Hey,

I'm trying to get an overall avg vs grouped avg aggregation in DevTools:

GET index1/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "exists": {
            "field": "my_measurement"
          }
        },
        {
          "exists": {
            "field": "my_measurement_weight"
          }
        }
      ]
    }
  }, 
  "aggs": {
    "location_agg": {
      "terms": {
        "field": "key.location.keyword"
      },
      "aggs": {
        "overall_avg_by_location": {
          "weighted_avg": {
            "value": {"field": "my_measurement"},
            "weight": {"field": "my_measurement_weight"}
          }
        },
        "device_location_agg": {
          "terms": {
            "field": "key.device.keyword"
          },
          "aggs": {
            "overall_avg_by_location_and_device": {
              "weighted_avg": {
                "value": {"field": "my_measurement"},
                "weight": {"field": "my_measurement_weight"}
              }
            }
          }
        },
        "KeepBadDevices": {
          "bucket_selector": {
            "buckets_path": {
              "overall_avg_by_location": "overall_avg_by_location.value",
              "overall_avg_by_location_and_device": "device_location_agg.value"
            },
            "script": "params.overall_avg_by_location > 20"
          }
        }
      }
    }
  }
}

When executing the request I'm getting the exception:

"type" : "illegal_argument_exception",
      "reason" : "No aggregation found for path [device_location_agg.overall_avg_by_location_and_device.value]

Is there a way to access the overall_avg_by_location_and_device nested aggregation so I won't be getting this error?

[spinscale (Alexander Reelsen)]

Take a look at the buckets_path syntax at Pipeline aggregations | Elasticsearch Guide [7.13] | Elastic - it's a little different notation using > instead of . at some positions (including the reasoning for that).

A small reproducible example with a few sample documents might help as well, otherwise this is mostly guessing

[Roy_Levy]

Ok, that's great to know about this way of accessing sibling aggs! I've changed the KeepBadDevices to:

"KeepBadDevices": {
          "bucket_selector": {
            "buckets_path": {
              "overall_avg_by_location": "overall_avg_by_location.value",
              "overall_avg_by_location_and_device": "device_location_agg>overall_avg_by_location_and_device"
            },
            "script": "params.overall_avg_by_location > 20"
          }
        }

And it access it successfully, but now I'm getting the following error for it:

"aggregation_execution_exception",
      "reason" : "buckets_path must reference either a number value or a single value numeric metric aggregation, got: [Object[]] at aggregation [device_location_agg]"

It seems to be a list of Objects. Is there any way to access it in this query? Or unpack the object?

[spinscale (Alexander Reelsen)]

again a reproducible example would be of great use, because this would explain the document structure and the aggs structure much better than any error message could

[Roy_Levy]

Hey, here is an example for the documents in the index which I'm trying to aggregate:

{
  "_index": "index1",
  "_type": "_doc",
  "_id": "abc",
  "_version": 1,
  "_score": null,
  "_source": {
    "key": {
      "location": "LA",
      "device": "IPHONE"
    },
    "my_measurement": 0.8,
    "my_measurement_weight": 352
},
{
  "_index": "index1",
  "_type": "_doc",
  "_id": "abd",
  "_version": 1,
  "_score": null,
  "_source": {
    "key": {
      "location": "UA",
      "device": "TABLET"
    },
    "my_measurement": 0.7,
    "my_measurement_weight": 276
}

Hope this explains the situation better
Again, the problem is that I'm getting an object type instead single value. Is there anything I'm missing here?

[spinscale (Alexander Reelsen)]

You are trying to access a value within another buckets array from the device_location_agg.buckets field. That one is not uniquely addressed...