Circuit_breaking_exception

We recently upgraded our elastic cloud deployment from 6.8.5 to 7.9
After the upgrade we are seeing the following error time to time.

{
  "error" : {
    "root_cause" : [
      {
        "type" : "circuit_breaking_exception",
        "reason" : "[parent] Data too large, data for [<http_request>] would be [416906520/397.5mb], which is larger than the limit of [408420352/389.5mb], real usage: [416906520/397.5mb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=32399/31.6kb, in_flight_requests=0/0b, model_inference=0/0b, accounting=4714192/4.4mb]",
        "bytes_wanted" : 416906520,
        "bytes_limit" : 408420352,
        "durability" : "PERMANENT"
      }
    ],
    "type" : "circuit_breaking_exception",
    "reason" : "[parent] Data too large, data for [<http_request>] would be [416906520/397.5mb], which is larger than the limit of [408420352/389.5mb], real usage: [416906520/397.5mb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=32399/31.6kb, in_flight_requests=0/0b, model_inference=0/0b, accounting=4714192/4.4mb]",
    "bytes_wanted" : 416906520,
    "bytes_limit" : 408420352,
    "durability" : "PERMANENT"
  },
  "status" : 429
}

This deployment consists of only one node with 1G memory. We would like to know the cause of this error. Is it due to the upgrade?

Thank you.

Yes, Elasticsearch 7.x has several improvements for node resiliency, you can find details here: https://www.elastic.co/blog/improving-node-resiliency-with-the-real-memory-circuit-breaker

I hope this helps!

1 Like

Where are you seeing this, Kibana?

Hi,
I see this error when I issue a simple curl command to any of the indices.
Regards,
Sajeeva.

What is the request?

Almost all queries. Even for something like this

curl -X GET <url>/index_name

You seem to have a very small heap. How much data do you have in the cluster?

Data size is close to 1G.
Yes, memory utilization is very high, but we did not receive this error when we were on 6.8.5. We started seeing these after upgrading to 7.9

Which exact version are you on? I recall reading something about a memory leak in 7.9.0 that I believe was fixed in 7.9.1.

Hi, do you have any sources to communicate on this memory leak problem that you mention?

Hi,
I was on 7.9.0, and I just upgraded it to 7.9.1. I will check the status and let you know.
Thank you for your help.