Parent circuit breaker calculation seems to be wrong with version 7.x

Elastic Stack Elasticsearch
21

The java version information is in my previous message:

openjdk 11.0.4 2019-07-16
OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-116.04.1)
OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-116.04.1, mixed mode, sharing)

We are currently running ES 7.3.0, and previously we were running 6.8.X.
For our 6.8.X install we were using our Java 11 install on the servers. If it helps, we are running in the Google Cloud.

22 
   We exactly facing the same issue,its annoying . Our production clusters are running with ES version 7.1.1 with OpenJDK 12.0.1(bundled version) with 10G of heap allocated. looking at discussion above it seems the problem is with JDK 12 . Also yesterday we have upgraded our Non-prod cluster  from 7.1.1 to 7.4.0 and that comes with JDK 13.

Also seeing from support matrix - https://www.elastic.co/support/matrix#matrix_jvm JDK 12 has been removed for ES version 7.4 which makes sense.

Question is, Does any one know if this issue gets resolved with JDK 13 ? for the past one day cluster seems to be fine and we didn't see any trips but if anyone has additional info with JDK 13, that would be a good information.

23

Hi @pjaikumar,

several other users who have run into this were using G1GC. Is that also the case for your cluster?

If so, the following change is important:


It is worthwhile double checking that the G1 settings are correct compared to that PR, i.e., that you have:

-XX:G1ReservePercent=25
-XX:InitiatingHeapOccupancyPercent=30

in your jvm.options file.

If you are not using G1 or you are using G1 with above settings, I would be interested in seeing your GC logs from Elasticsearch (together with exception info from log file) if this should reoccur.

1 Like
24

Hi @HenningAndersen
Yes correct we were using G1 with IHOP of 75%. Today we changed it to CMS with below default settings and running a test against it, So far the results look pretty good and we haven't seen any Circuit breaker errors.

-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

1 Like