Circuit_breaking_exception for [request] Data too large, but index being queried is only 8.44mb

I'm getting this error when running the aggregation below. The index being queried is only 8.44mb. I was able to run these types of queries on a much smaller cluster, in terms of RAM, with ES v6 and never see these types of aggregation errors. On ES 8.3.3 my logs are full of aggregation errors, always on [request]. Sometimes the aggregation will work, but most of the time they will not.

Obviously, I'm missing some key concept here with aggregations, but I just need to know how to reconfigure my Elastic Cloud cluster so aggregations work again without any of these types of exceptions.

In the screenshots below, you can see I'm not having JVM pressure issues and the index being queried is super tiny.

I've tried running POST _cache/clear?fielddata=true to no avail and I've read through the docs, but I frankly don't get what has changed from ES v6.3 and ES v8.3.3 for aggregations. Running the same aggregations with half the cluster memory in ES v6.3 I never had ES crash even once due to JVM heap issues. I'm sure this is some type of configuration issue, but I've reading other circuit breaker exception posts and I cannot yet sort this riddle for my use case.

GET _cat/nodes?v=true&h=name,node*,heap*
name                  id   node.role heap.current heap.percent heap.max
instance-0000000001   J9uc himrst           3.1gb           42    7.5gb
instance-0000000000   1bLn himrst           2.6gb           35    7.5gb
tiebreaker-0000000002 XjWb mv             322.2mb           62    512mb
{
  "error": {
    "root_cause": [],
    "type": "search_phase_execution_exception",
    "reason": "",
    "phase": "fetch",
    "grouped": true,
    "failed_shards": [],
    "caused_by": {
      "type": "circuit_breaking_exception",
      "reason": "[request] Data too large, data for [<reduce_aggs>] would be [11892235471/11gb], which is larger than the limit of [4831838208/4.5gb]",
      "bytes_wanted": 11892235471,
      "bytes_limit": 4831838208,
      "durability": "TRANSIENT"
    }
  },
  "status": 429
}
GET vanguard_products/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "tags": "West"
          }
        }
      ],
      "filter": {
        "terms": {
          "sku.exact": [
            "2153700",
            "2153702",
            "7150100",
            "7150500",
            "7150200",
            "7150300"
          ]
        }
      }
    }
  },
  "aggs": {
    "fulfillment_skus": {
      "cardinality": {
        "field": "sku.exact"
      }
    }
  }
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.