# Index failed to prepare/warm -\> Cause IO burst & high latency

**URL:** <https://discuss.elastic.co/t/index-failed-to-prepare-warm-cause-io-burst-high-latency/11292>\
**Category:** Elasticsearch\
**Created:** [March 25, 2013, 2:21pm UTC](https://discuss.elastic.co/t/index-failed-to-prepare-warm-cause-io-burst-high-latency/11292 "2013-03-25T14:21:41Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\
**Post date:** [March 25, 2013, 2:21pm UTC](https://discuss.elastic.co/t/index-failed-to-prepare-warm-cause-io-burst-high-latency/11292/1 "2013-03-25T14:21:41Z")

</div>

Hi,

We are running a quite large cluster of ElasticSearch servers in  
production, we see from time to time error like this;

[2013-03-25 08:47:16,860][WARN][index.engine.robin] [es37b]  
[INDEX\_NAME][178] failed to prepare/warm  
org.elasticsearch.common.util.concurrent.EsRejectedExecutionException:  
rejected execution of  
[org.elasticsearch.indices.warmer.InternalIndicesWarmer$1]

always followed by a burst of IO, and the shard being not available (plus  
the search queue being filled) for a time, because of the IO wait.  
Basically, this cause my response times to go through the rough for some  
time too..

My guess is that because of that, the FS cache is being invalidated and it  
has to refill it from disk... I don't know if it's a bug or what can cause  
this. My index is quite large (~9Billions documents, 250 shards, 0  
replicas, ~20gb per shards), I am fully aware that with replicas the  
problem should solve by itself, of at least being mitigated, but for now we  
haven't enough IO to afford it. I am wondering if there is some way to at  
least mitigate this issue for the time being.

Thanks

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:44am UTC](https://discuss.elastic.co/t/index-failed-to-prepare-warm-cause-io-burst-high-latency/11292/2 "2017-07-06T02:44:45Z")

</div>

Closed on July 6, 2017, 2:44am UTC