# 6.5.0 JVM Crash

**URL:** <https://discuss.elastic.co/t/6-5-0-jvm-crash/161620>\
**Category:** Elasticsearch\
**Created:** [December 20, 2018, 5:30am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620 "2018-12-20T05:30:34Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [December 20, 2018, 5:30am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/1 "2018-12-20T05:30:34Z")

</div>

Hi. Running Elasticsearch 6.5.0 in Docker. Been upgrading our environment and so replacing Elastic servers. Everything was going fine until one of our servers started JVM crashing.

Here's the JVM output: [https://gist.github.com/bitva77/d8e47b894263525fcc6fcb8c2755eb90](https://gist.github.com/bitva77/d8e47b894263525fcc6fcb8c2755eb90)

Running in Docker (18.0.6)

Really have no idea what to do about this.

Any insight would be appreciated.

Thanks!

-Matt

---

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [December 20, 2018, 5:58am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/2 "2018-12-20T05:58:22Z")

</div>

Whether sharding was enabled or not, ES still crashed and would loop (docker restarts the container but it crashes again). Server reboot had no effect.

Red Hat 7.5.

I've since destroyed that server and have built a new one, identical and so far so good (knocking on wood).

~6TB cluster size

3 Master  
2 Data

It's the 3rd master that crashed. I have the new one in and the cluster is currently doing it's replication thing.

Will update this thread if things go sound again

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 25, 2018, 6:39am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/3 "2018-12-25T06:39:33Z")

</div>

What is the heap size of your data nodes? How many indices/shards do you have in the cluster?

---

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [December 26, 2018, 5:55pm UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/4 "2018-12-26T17:55:34Z")

</div>

26GB heap with 64GB RAM.

239 indices plus 58 "." indices (so 297 total). There are like 30 closed.

4,880 shards.

The number of shards will go down a bit. We were at like 10\*2 across the board but have since changed the template to either 4 or 8 \*2.

Thanks!

---

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [January 15, 2019, 9:57pm UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/5 "2019-01-15T21:57:23Z")

</div>

Got the JVM crashing again, in a loop:

> <https://gist.github.com/bitva77/1a29600faa3c711175cbd082cf6563e7>

---

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [January 18, 2019, 1:57am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/6 "2019-01-18T01:57:14Z")

</div>

No matter what I try it crashes now: OpenJDK, OracleJDK, docker image, no docker image:

> <https://gist.github.com/bitva77/6710c9b7f97be56752bb2cb3c33236af>

Anything else you need. I guess nobody cares until we pay ya'll money (which we're working on) but any help you can provide now would be great.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 18, 2019, 6:59am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/7 "2019-01-18T06:59:46Z")

</div>

Are using [our official docker image](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html)? How much resources are you giving the image?

---

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [January 18, 2019, 7:06am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/8 "2019-01-18T07:06:55Z")

</div>

Both actually. And once with the tar.gz file....

---

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [January 18, 2019, 8:30am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/9 "2019-01-18T08:30:38Z")

</div>

Oh sorry, resource wise:

26GB heap on a 64GB server with 8 cores

---

<div class="post-metadata">

**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\
**Post date:** [January 21, 2019, 12:43am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/10 "2019-01-21T00:43:49Z")

</div>

Happens with the vanilla image as well. This is Red Hat 7.6.

All I'm running is:

`docker run docker.elastic.co/elasticsearch/elasticsearch:6.5.4`

it sits for about a minute before crashing

> <https://gist.github.com/bitva77/d887d45cb4d877812a82ce2385eb687e>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 18, 2019, 12:43am UTC](https://discuss.elastic.co/t/6-5-0-jvm-crash/161620/11 "2019-02-18T00:43:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
