Circuit breaking EXception while using Reindex API

Elastic Stack Elasticsearch
1

Hi,

while using Reindex API we are getting circuit breaking Exception

  "cause" : {
    "type" : "circuit_breaking_exception",
    "reason" : "[parent] Data too large, data for [<transport_request>] would be [3880571978/3.6gb], which is larger than the limit of [3759774105/3.5gb], real usage: [3877310992/3.6gb], new bytes reserved: [3260986/3.1mb], usages [request=0/0b, fielddata=0/0b, in_flight_requests=3260986/3.1mb, accounting=1243875/1.1mb]",
    "bytes_wanted" : 3880571978,
    "bytes_limit" : 3759774105,
    "durability" : "TRANSIENT"
  },
  "status" : 429
},
2

Hi @Siva_Karan,

could you share the jvm.options you're using? (the GC settings and heap sizing is what I'm looking for in particular).

3

Hi @Armin_Braun

please look this jvm options we are using

export ES_JAVA_OPTS="-XX:HeapDumpPath=<<<_LOG_PATH/$NODE_NAME-heapdump.hprof -Xloggc:<<<<_LOG_PATH/gc.log -XX:+UseConcMarkSweepGC -Xms8096m -Xmx8096m -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:MaxTenuringThreshold=6 -XX:+PrintTenuringDistribution -XX:SurvivorRatio=6 -XX:+CMSClassUnloadingEnabled"

4

@Siva_Karan thanks for those numbers.

You seem to be using non-standard limits in the real-memory circuit breaker (i.e. less than 50% of total heap seem to be the limit for real memory circuit breaking)?
What did you set the limits to? I think increasing these limits might help you make the circuit breaker less likely to trigger.