# Upgrading/changing JVM

**URL:** <https://discuss.elastic.co/t/upgrading-changing-jvm/10095>\
**Category:** Elasticsearch\
**Created:** [December 17, 2012, 4:16pm UTC](https://discuss.elastic.co/t/upgrading-changing-jvm/10095 "2012-12-17T16:16:49Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dan\_Fairs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_fairs/32/1205_2.png) [@Dan\_Fairs](https://discuss.elastic.co/u/Dan_Fairs)\
**Post date:** [December 17, 2012, 4:16pm UTC](https://discuss.elastic.co/t/upgrading-changing-jvm/10095/1 "2012-12-17T16:16:49Z")

</div>

Hi,

What's the best approach for upgrading the JVM used by a cluster?

We're currently using the OpenJDK as packaged with Ubuntu 12.04 (1.6.0\_24, fwiw), but I do keep reading that Oracle's latest Java 7 is the way to go. Obviously I'll test any upgrades on a test cluster first, but it'd be good to know what direction I should head in!

Can I upgrade the JVM on one machine at a time, and basically do a rolling restart of the cluster? Would you expect anything to break if different nodes are using different JVM versions, or using data files created from ES on the older JVM with a newer one?

I'm using ES 0.19.8.

Cheers,  
Dan

--  
Dan Fairs | [dan.fairs@gmail.com](mailto:dan.fairs@gmail.com) | @danfairs | [secondsync.com](http://secondsync.com)

--

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [December 17, 2012, 7:59pm UTC](https://discuss.elastic.co/t/upgrading-changing-jvm/10095/2 "2012-12-17T19:59:05Z")

</div>

Hi Dan,

it's tricky. I know there are some installations that run with mixed Java,  
Version 6 and 7, for a significant time without issues. Note, this can  
work, but you should avoid administrative tasks like creating indices,  
adding new plugins, and indexing.

A rolling update should work as long as you run only the search API. You  
should take care the index is "silent", that is, flushed to the gateway  
with no outstanding writes, and no indexing.

I recommended a test: start another cluster instance with a mixed Java 6  
and a Java 7 node on a test machine, copy your configuration to them and  
start indexing and search. Check if the cluster works.

Due to subtle serialization issues, your ES cluster will not be able to  
transport internal Java exceptions in bulk index responses between  
different Java 6 and 7 JVM nodes. It is possible you will be affected by  
this if you allow the bulk index API while you do the update.

Regarding data files, there are a lot of JVM issues the Lucene committers  
are aware of. There had been suggestions a few months before to even block  
Lucene from running on blacklisted JVMs. See also  
[[LUCENE-4276] refuse to execute on broken corrupting jvms - ASF JIRA](https://issues.apache.org/jira/browse/LUCENE-4276) In summary, there are  
some JVMs that are known to harm your index when writing to it, but it is  
not easy to identify the version, especially in Ubuntu's Java numbering.  
But don't be worried too much, I think the best alternative is to move to  
the latest Java 7 JVM. Oracle/OpenJDK are quite aligned in the Java 7  
releases (except security fixes unfortunately).

If it helps, when I moved to Java 7, I took the whole cluster down, because  
I found it is the fastest method to get the cluster up again.

Cheers,

Jörg

On Monday, December 17, 2012 5:16:49 PM UTC+1, Dan Fairs wrote:

> Hi,
> 
> What's the best approach for upgrading the JVM used by a cluster?
> 
> We're currently using the OpenJDK as packaged with Ubuntu 12.04 (1.6.0\_24,  
> fwiw), but I do keep reading that Oracle's latest Java 7 is the way to go.  
> Obviously I'll test any upgrades on a test cluster first, but it'd be good  
> to know what direction I should head in!
> 
> Can I upgrade the JVM on one machine at a time, and basically do a rolling  
> restart of the cluster? Would you expect anything to break if different  
> nodes are using different JVM versions, or using data files created from ES  
> on the older JVM with a newer one?
> 
> I'm using ES 0.19.8.
> 
> Cheers,  
> Dan
> 
> --  
> Dan Fairs | [dan....@gmail.com](mailto:dan....@gmail.com) \<javascript:\> | @danfairs | [secondsync.com](http://secondsync.com)

--

---

<div class="post-metadata">

**Author:** ![Dan\_Fairs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dan_fairs/32/1205_2.png) [@Dan\_Fairs](https://discuss.elastic.co/u/Dan_Fairs)\
**Post date:** [December 19, 2012, 4:36pm UTC](https://discuss.elastic.co/t/upgrading-changing-jvm/10095/3 "2012-12-19T16:36:50Z")

</div>

Hi Jörg,

Thanks for detailed reply!

> A rolling update should work as long as you run only the search API. You should take care the index is "silent", that is, flushed to the gateway with no outstanding writes, and no indexing.
> 
> I recommended a test: start another cluster instance with a mixed Java 6 and a Java 7 node on a test machine, copy your configuration to them and start indexing and search. Check if the cluster works.

Right - this was basically my plan.

> Due to subtle serialization issues, your ES cluster will not be able to transport internal Java exceptions in bulk index responses between different Java 6 and 7 JVM nodes. It is possible you will be affected by this if you allow the bulk index API while you do the update.

Mm, almost all our index operations are bulk actually; we add up to a million documents per day.

> Regarding data files, there are a lot of JVM issues the Lucene committers are aware of. There had been suggestions a few months before to even block Lucene from running on blacklisted JVMs. See also [[LUCENE-4276] refuse to execute on broken corrupting jvms - ASF Jira](https://issues.apache.org/jira/browse/LUCENE-4276) In summary, there are some JVMs that are known to harm your index when writing to it, but it is not easy to identify the version, especially in Ubuntu's Java numbering. But don't be worried too much, I think the best alternative is to move to the latest Java 7 JVM. Oracle/OpenJDK are quite aligned in the Java 7 releases (except security fixes unfortunately).

Right. This was also my plan! I'm not really a Java person, so understanding the differences/similarities between the Oracle/OpenJDK releases is new to me.

> If it helps, when I moved to Java 7, I took the whole cluster down, because I found it is the fastest method to get the cluster up again.

I think that's what I'll do, then. It does seem the safest all round. And, of course, I'll be testing on our staging cluster.

Thanks again.

## Cheers, Dan

Dan Fairs | [dan.fairs@gmail.com](mailto:dan.fairs@gmail.com) | @danfairs | [secondsync.com](http://secondsync.com)

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:59am UTC](https://discuss.elastic.co/t/upgrading-changing-jvm/10095/4 "2017-07-06T02:59:15Z")

</div>


