Running into a CircuitBreakingException when doing a restore for Elasticsearch 7.7.1
Based on this issue, I suspect that I can try disabling indices.breaker.total.use_real_memory. Alternatively, I can try reducing CMSInitiatingOccupancyFraction. Are there any other recommendations? I don't have much experience with JVM tuning.
JVM version (java -version):
openjdk version "1.8.0_265"
OpenJDK Runtime Environment (build 1.8.0_265-8u265-b01-0ubuntu2~16.04-b01)
OpenJDK 64-Bit Server VM (build 25.265-b01, mixed mode)
ES Log:
Failed to complete action: restore. <class 'curator.exceptions.FailedExecution'>: Exception encountered. Rerun with loglevel DEBUG and/or check Elasticsearch logs for more information. Exception: Unable to obtain recovery information for specified indices. Error: TransportError(429, 'circuit_breaking_exception', '[parent] Data too large, data for [<http_request>] would be [12543328208/11.6gb], which is larger than the limit of [12240656793/11.3gb], real usage: [12543328208/11.6gb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=0/0b, in_flight_requests=0/0b, accounting=1592100/1.5mb]')
jvm.option:
-Xms12g
-Xmx12g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+AlwaysPreTouch
-Xss1m
-XX:-OmitStackTraceInFastThrow
8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:/var/log/elasticsearch/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m