Invalid term order aggregation path

[SJLG]

Hi

In our software we use the following terms aggregation to group the records by the field sGroup. Each record has a timestamp lTsRequest, which we use to retrieve the most recent / latest values of some other field for each group.

POST /myFancyIndex/_search?size=0
{
  "aggs": {
    "group": {
      "terms": {
        "field": "sGroup.keyword"
      },
      "aggs": {
        "most_recent_status": {
          "terms": {
            "field": "iStatus",
            "size" : 1
          },
          "aggs": {
            "most_recent_timestamp": {
              "top_hits": {
                "size": 1,
                "sort": [
                  {
                    "myTimestamp": {
                      "order": "desc"
                    }
                  }
                ]
              }
            }
          }
        }
      }
    }
  }
}

This works like a charm. Now we wish to order the results by their most recent status, which we implemented using

"order" : {
   "most_recent_status > most_recent_timestamp" : "desc"
}

defined in the group aggregation. ES reponds with

Invalid terms aggregation order path [most_recent_status>most_recent_timestamp]. Terms buckets can only be sorted on a sub-aggregator path that is built out of zero or more single-bucket aggregations within the path and a final single-bucket or a metrics aggregation at the path end. Sub-path [most_recent_status] points to non single-bucket aggregation

Why does this happen? Each aggregation on the path returns at most one bucket.

I appreciate your help and thank you in advance.

Cheers
Simon

[polyfractal (Zachary Tong)]

@SJLG

Why does this happen? Each aggregation on the path returns at most one bucket.

Yeah, that error is a bit confusing because it's using terms that are internal to ES. Inside Elasticsearch, there are multi-bucket aggs like terms, date_histo, etc which return multiple buckets. And there are single-bucket aggs, like filter which always return a single bucket.

The error message is referring to that; the type of agg, not necessarily how many buckets are actually being returned based on your size param.

We could probably improve the code to allow pathing into multi-bucket aggs when they only return one bucket due to an explicit size, but today it doesn't have that capability.