# Recovering from loop on "java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot flush"

**URL:** <https://discuss.elastic.co/t/recovering-from-loop-on-java-lang-illegalstateexception-this-writer-hit-an-outofmemoryerror-cannot-flush/4086>\
**Category:** Elasticsearch\
**Created:** [March 13, 2011, 9:49am UTC](https://discuss.elastic.co/t/recovering-from-loop-on-java-lang-illegalstateexception-this-writer-hit-an-outofmemoryerror-cannot-flush/4086 "2011-03-13T09:49:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![charles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/charles/32/3254_2.png) [@charles](https://discuss.elastic.co/u/charles)\
**Post date:** [March 13, 2011, 9:49am UTC](https://discuss.elastic.co/t/recovering-from-loop-on-java-lang-illegalstateexception-this-writer-hit-an-outofmemoryerror-cannot-flush/4086/1 "2011-03-13T09:49:09Z")

</div>

Howdy!

I recently had a failure in my log-indexing cluster wherein  
ElasticSearch ceased reading from its RabbitMQ river, and has been  
continually giving RefreshFailedExceptions (showing an  
IllegalStateException, caused by an OutOfMemoryError) as the cause.

When in this state, the JVM does not shut itself down properly on  
SIGTERM.

The error itself doesn't bother me nearly so much as not having an  
automated means to recover from it; if I could configure RabbitMQ to  
always exit in such cases, for instance, the outer process supervision  
layer could restart it.

Thoughts?

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [March 14, 2011, 9:56am UTC](https://discuss.elastic.co/t/recovering-from-loop-on-java-lang-illegalstateexception-this-writer-hit-an-outofmemoryerror-cannot-flush/4086/2 "2011-03-14T09:56:35Z")

</div>

Yea, thats sadly how Java works. There isn't a nice way to hook into the JVM and be notified when it runs out of memory (and the exception can basically happen anywhere, and might actually be trapped).

The service wrapper tries to help with that in allowing to configure a filter that will restart the Java process if OutOfMemory error appears in the logs it tracks.  
On Sunday, March 13, 2011 at 11:49 AM, Charles Duffy wrote:  
Howdy!

> I recently had a failure in my log-indexing cluster wherein  
> Elasticsearch ceased reading from its RabbitMQ river, and has been  
> continually giving RefreshFailedExceptions (showing an  
> IllegalStateException, caused by an OutOfMemoryError) as the cause.
> 
> When in this state, the JVM does not shut itself down properly on  
> SIGTERM.
> 
> The error itself doesn't bother me nearly so much as not having an  
> automated means to recover from it; if I could configure RabbitMQ to  
> always exit in such cases, for instance, the outer process supervision  
> layer could restart it.
> 
> Thoughts?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:10am UTC](https://discuss.elastic.co/t/recovering-from-loop-on-java-lang-illegalstateexception-this-writer-hit-an-outofmemoryerror-cannot-flush/4086/3 "2017-07-06T04:10:08Z")

</div>


