# Failed to flush shard on translog threshold

**URL:** <https://discuss.elastic.co/t/failed-to-flush-shard-on-translog-threshold/17175>\
**Category:** Elasticsearch\
**Created:** [April 24, 2014, 1:12pm UTC](https://discuss.elastic.co/t/failed-to-flush-shard-on-translog-threshold/17175 "2014-04-24T13:12:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Drew\_Blessing](https://avatars.discourse-cdn.com/v4/letter/d/838e76/32.png) [@Drew\_Blessing](https://discuss.elastic.co/u/Drew_Blessing)\
**Post date:** [April 24, 2014, 1:12pm UTC](https://discuss.elastic.co/t/failed-to-flush-shard-on-translog-threshold/17175/1 "2014-04-24T13:12:39Z")

</div>

We have been fighting with this issue for several months now. We've had it  
occur in 0.90.6 - 0.90.13 and now in 1.1, too. It went away for a while  
after we upgraded to 1.1 so we thought we had beaten it, but it cropped up  
again last night. Any thoughts or help appreciated.

The issue seems to occur most often during or immediately after we run an  
index. It always happens on the exact same index, too. One, maybe two,  
shards will fail and queries against that index will fail from then on.  
We've poured through the code that indexes this data and can't see  
anything. We also don't know why client code would be able to kill a  
server. All data nodes in the cluster are CentOS 6.5 servers running Java  
1.7.0u51. All non-data clients are using a java client library. There is 1  
non-data node for the client that loads data and 2 more non-data nodes  
serving the front-end. Let me know if there is any other information you  
think would be pertinent. Thanks for your help.

[2014-04-23 15:43:14,141][INFO][cluster.metadata]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product] update\_mapping [product] (dynamic)  
[2014-04-23 15:46:00,040][INFO][cluster.metadata]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product] update\_mapping [product] (dynamic)  
[2014-04-23 15:47:26,884][WARN][transport]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] Received response for a request that has timed  
out, sent [31301ms] ago, timed out [1300ms] ago, action  
[discovery/zen/fd/ping], node  
[[[servicebus-PROD-tomcatapps3.example.com](http://servicebus-PROD-tomcatapps3.example.com)][ewvSIKmYRfS2qKiojcUw7w][[tomcatapps3.example.com](http://tomcatapps3.example.com)][inet[/10.0.0.119:9300]]{client=true,  
data=false, local=false}], id [160544]  
[2014-04-23 16:30:38,734][INFO][cluster.service]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] removed  
{[[servicebus-PROD-tomcatapps3.example.com](http://servicebus-PROD-tomcatapps3.example.com)][ewvSIKmYRfS2qKiojcUw7w][[tomcatapps3.example.com](http://tomcatapps3.example.com)][inet[/10.0.0.119:9300]]{client=true,  
data=false, local=false},}, reason:  
zen-disco-node\_left([[servicebus-PROD-tomcatapps3.example.com](http://servicebus-PROD-tomcatapps3.example.com)][ewvSIKmYRfS2qKiojcUw7w][[tomcatapps3.example.com](http://tomcatapps3.example.com)][inet[/10.0.0.119:9300]]{client=true,  
data=false, local=false})  
[2014-04-23 16:30:49,756][INFO][cluster.service]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] added  
{[[servicebus-PROD-tomcatapps3.example.com](http://servicebus-PROD-tomcatapps3.example.com)][ixCfue8YRMq3XhOOcdlmEg][[tomcatapps3.example.com](http://tomcatapps3.example.com)][inet[/10.0.0.119:9300]]{client=true,  
data=false, local=false},}, reason: zen-disco-receive(join from  
node[[[servicebus-PROD-tomcatapps3.example.com](http://servicebus-PROD-tomcatapps3.example.com)][ixCfue8YRMq3XhOOcdlmEg][[tomcatapps3.example.com](http://tomcatapps3.example.com)][inet[/10.0.0.119:9300]]{client=true,  
data=false, local=false}])  
[2014-04-23 16:35:51,054][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][4] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][4]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:35:54,599][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][1] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][1]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:02,389][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][1] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][1]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:06,758][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][4] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][4]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:11,683][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][1] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][1]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:14,900][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][4] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][4]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:23,382][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][4] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][4]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:24,811][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][1] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][1]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:28,577][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][4] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][4]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more  
[2014-04-23 16:36:41,739][WARN][index.translog]  
[[servicebus-es1.example.com](http://servicebus-es1.example.com)] [product][1] failed to flush shard on translog  
threshold  
org.elasticsearch.index.engine.FlushFailedEngineException: [product][1]  
Flush failed  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:826)  
at  
org.elasticsearch.index.shard.service.InternalIndexShard.flush(InternalIndexShard.java:589)  
at  
org.elasticsearch.index.translog.TranslogService$TranslogBasedFlush$1.run(TranslogService.java:194)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)  
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader  
is closed  
at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:253)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:102)  
at  
org.apache.lucene.index.CompositeReader.getContext(CompositeReader.java:56)  
at org.apache.lucene.index.IndexReader.leaves(IndexReader.java:503)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.contains(DeleteByQueryWrappingFilter.java:122)  
at  
org.elasticsearch.index.search.child.DeleteByQueryWrappingFilter.getDocIdSet(DeleteByQueryWrappingFilter.java:81)  
at  
org.elasticsearch.common.lucene.search.ApplyAcceptedDocsFilter.getDocIdSet(ApplyAcceptedDocsFilter.java:45)  
at  
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:142)  
at  
org.apache.lucene.search.FilteredQuery$RandomAccessFilterStrategy.filteredScorer(FilteredQuery.java:526)  
at org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:136)  
at  
org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:546)  
at  
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:284)  
at  
org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3164)  
at  
org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3155)  
at  
org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2874)  
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)  
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)  
at  
org.elasticsearch.index.engine.internal.InternalEngine.flush(InternalEngine.java:812)  
... 5 more

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/533015d6-3757-40c1-9eac-a3ef61a4f46b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/533015d6-3757-40c1-9eac-a3ef61a4f46b%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:33am UTC](https://discuss.elastic.co/t/failed-to-flush-shard-on-translog-threshold/17175/2 "2017-07-06T01:33:45Z")

</div>