# Elasticsearch 7.16.1 crashing randomly

**URL:** <https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827>\
**Category:** Elasticsearch\
**Created:** [December 14, 2021, 3:49pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827 "2021-12-14T15:49:37Z")\
**Posts on this page:** 9\
**Page:** 2

<div class="post-metadata">

**Author:** ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)\
**Post date:** [December 21, 2021, 3:34pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/22 "2021-12-21T15:34:27Z")

</div>

Without the verbose logs, I don't think that gets us any closer. It sounds like you were able to reproduce it though? Do you have step-by-step instructions that someone else could follow (ideally exact commands to run)?

---

<div class="post-metadata">

**Author:** ![joschi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joschi/32/44946_2.png) [@joschi](https://discuss.elastic.co/u/joschi)\
**Post date:** [December 21, 2021, 6:08pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/23 "2021-12-21T18:08:46Z")

</div>

In our case it turned out that an APM agent instrumenting the Elasticsearch node was the culprit, which didn't "know" about the move of the `XContent` class in ES 7.16.0 yet.

Sorry for the false alarm and thanks for looking into this! ❤

---

<div class="post-metadata">

**Author:** ![ppuschmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppuschmann/32/146751_2.png) [@ppuschmann](https://discuss.elastic.co/u/ppuschmann)\
**Post date:** [December 22, 2021, 9:04am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/24 "2021-12-22T09:04:12Z")

</div>

We're also running some APM-agent that instruments JVM-processes dynamically.  
Regarding this issue I didn't do the connection to the APM-agent before.

---

<div class="post-metadata">

**Author:** ![ppuschmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppuschmann/32/146751_2.png) [@ppuschmann](https://discuss.elastic.co/u/ppuschmann)\
**Post date:** [December 22, 2021, 9:15am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/25 "2021-12-22T09:15:14Z")

</div>

Please take a look into these logs: [Elasticsearch 7.16.2 verbose logs · GitHub](https://gist.github.com/ppuschmann/2a1c534944c5a9e6ebfee0124c54c16b)

These are the last I could create/trigger.

---

<div class="post-metadata">

**Author:** ![ppuschmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppuschmann/32/146751_2.png) [@ppuschmann](https://discuss.elastic.co/u/ppuschmann)\
**Post date:** [December 22, 2021, 2:15pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/27 "2021-12-22T14:15:10Z")

</div>

The inital logs were from different nodes.

But [Elasticsearch 7.16.1 crashing randomly - #21 by ppuschmann](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/21) matches to these verbose-logs.

Should I look for something in (up-to-date) application-logs?

---

<div class="post-metadata">

**Author:** ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)\
**Post date:** [December 22, 2021, 2:55pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/28 "2021-12-22T14:55:15Z")

</div>

The only thing that stands out to me is /tmp/.instana/sensor-Elasticsearch-javaagent-1.1.59-29035-6754744723716598642.jar. I can't find a copy of that jar or the source code, but it looks like Instana injects code for the sake of monitoring? Maybe they have compiled against an old version of Elasticsearch? I'm just guessing though. Can you disable that and see if the problem is reproducible?

---

<div class="post-metadata">

**Author:** ![ppuschmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppuschmann/32/146751_2.png) [@ppuschmann](https://discuss.elastic.co/u/ppuschmann)\
**Post date:** [December 22, 2021, 3:09pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/29 "2021-12-22T15:09:59Z")

</div>

Hi Keith, I will try to disable Instana and do some tests again.

---

<div class="post-metadata">

**Author:** ![ppuschmann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppuschmann/32/146751_2.png) [@ppuschmann](https://discuss.elastic.co/u/ppuschmann)\
**Post date:** [December 29, 2021, 10:23am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/30 "2021-12-29T10:23:36Z")

</div>

Everything is now looking fine.  
So it was an issue of the APM-tool, not Elasticsearch ☹ Sorry.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 26, 2022, 10:24am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827/31 "2022-01-26T10:24:34Z")

</div>

Closed on January 26, 2022, 10:24am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/elasticsearch-7-16-1-crashing-randomly/291827.md?page=1)
