Circuit breaker with durability permanent auto fixed?

Where do I find out what kinds of errors lead to a transient circuit breaker exceptions vs permanent?

Yesterday we got this exception, but it auto resolved surprisingly? Won't permanent exceptions require some sort of manual intervention like restart?

{
  "error": {
    "root_cause": [
      {
        "type": "circuit_breaking_exception",
        "reason": "[parent] Data too large, data for [<transport_request>] would be [32159782356/29.9gb], which is larger than the limit of [30601641984/28.5gb], real usage: [32159779328/29.9gb], new bytes reserved: [3028/2.9kb]",
        "bytes_wanted": 32159782356,
        "bytes_limit": 30601641984,
        "durability": "PERMANENT"
      }
    ],
    "type": "circuit_breaking_exception",
    "reason": "[parent] Data too large, data for [<transport_request>] would be [32159782356/29.9gb], which is larger than the limit of [30601641984/28.5gb], real usage: [32159779328/29.9gb], new bytes reserved: [3028/2.9kb]",
    "bytes_wanted": 32159782356,
    "bytes_limit": 30601641984,
    "durability": "PERMANENT"
  }
}

From the log above, it is weird that it estimated 32159782356 bytes whereas heap usage never went above 18gb. We have 30gb data nodes. Can someone explain the internals briefly?

Which circuit breaker is used for indexing? for searches?

Do we have a known list of requests that lead to circuit tripping? e.g. for above it was <transport_request>, i'd look to know all of them

Isn't it possible to know which nodes are tripped at any given point?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.