Circuit Breaking Exception

Elastic Stack Elasticsearch
1

I have recently migrated the cluster from 2.4 to 6.3.0 and I am getting circuit breaking exception every few hours on the nodes

Cluster details -
Data nodes : 13
Master nodes : 3
Query nodes : 3

We aren't using this cluster for searching in production as of now. Only indexing is happening on this cluster
{
"type" : "failed_node_exception",
"reason" : "Failed node [YMekd6kcQPydIHKnORSX6g]",
"caused_by" : {
"type" : "circuit_breaking_exception",
"reason" : "[parent] Data too large, data for [<transport_request>] would be [6608475256/6.1gb], which is larger than the limit of [6608348774/6.1gb]",
"bytes_wanted" : 6608475256,
"bytes_limit" : 6608348774
}
}

Heap size is 31 GB
I have overwritten this setting : indices.breaker.total.limit: 20%
With the same settings, indexing is working fine in 2.4.0, but failing in 6.3.0

Is this a bug in 6.3.0 version?

Thanks,
Nitish

2

Given the limit you have in place this seems expected. It sounds more like an issue with Elasticsearch 2.4 not triggering circuit breakers correctly. Why have you set indices.breaker.total.limit to such a low value?

3

Thanks for the explanation

I have set the value to 50% and restarted the failed nodes

Will post it here if I face this issue again

Thanks,
Nitish

4

Why are you overriding this in the first place? Why not use the default value?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.