I read in this doc this doc that CMS is still the recommended garbage collector for ES. That doc has a warning at the top that it is for ES 2.x and so might be out of date, though.
I'm running G1 on my - large heap & many CPU cores - production systems for quite some time. Did this after benchmarking on ES 6.0. G1 was significantly faster for my setup.
I'm also very interested in the status: no reply on my last question about this Reasons for not using G1 GC?. I cannot find a currently valid reason for not using G1. Especially interesting since G1 is the default CG for Java 9...
we will be supporting G1 with the upcoming version of Elasticsearch 6.5.0 but only if you run Elasticsearch on JDK 10 or later. For more context: the support has been added in #33685.
Because G1 stability left a lot to be desired in earlier JDK versions and we found serious bugs in G1. See e.g.: https://www.elastic.co/blog/a-heap-of-trouble. We take performance very seriously but if we know that such bugs cause index corruption it would be irresponsible and a disservice to our users if we would claim to support it.
These are arguments from 2016. Yes, back then CG1 was experimental, buggy and not recomendable. Since then (2016), the issues were fixed and since Java 9 (2017) G1 is the standard carbage collector. So G1 is not a experimental CG but the standard. So the 'buggy' argument is no longer valid.
Index corruption as the result of Jaba bugs: that was 2016 and if fixed. Why support G1 in Java 10+ but not in 8? I'm sure the G1 bugfixes are backported. Is this decision based on recent tests - and thus valid arguments in 2018 - or based on old data from 2016?
I think many professional parties run ES on LTS OSs. LTS OS versions typically support one major version over the lifetime of the OS. Java 11 is the first LTS Java since 8 and brand new. So it's not included in any LTS OS yet. That is why I think including offical G1 support for Java 8 by Elastic would be a big plus.
declaring that a particular configuration is supported is a very strong statement for us to make. It means that we are committing to deal with any issues found when running Elasticsearch in such a configuration, which we can only do once we are very confident there are no bugs or performance issues related to the configuration itself. A GC bug can have very serious consequences, including corruption and data loss, and Lucene and Elasticsearch are unusually stressful workloads for the JVM so they tend to find issues that other tests might miss. Moreover the existence of past bugs can be a good indicator of future bugs, so the issues that were fixed in 2016 still raise concerns today. It really does take years to gain the level of confidence needed to declare a configuration to be supported.
We are aware that some people, like yourself, are successfully running their production systems using G1GC (and other unsupported options) and these pioneers are part of the reason why we put the effort into choosing new configurations to test so we can declare them to be supported. However, the Java 8 G1GC implementation is different enough from the Java 10 implementation (and by extension, also the Java 11 implementation) that we must consider them to be separate things. Given that Java 8 is approaching the end of its life, it is difficult to justify the effort that would be required for us to declare support for the Java 8 G1GC too.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
