# Indexing/shard failure
**URL:** https://discuss.elastic.co/t/indexing-shard-failure/7178
**Category:** Elasticsearch
**Created:** [March 30, 2012, 1:04pm UTC](https://discuss.elastic.co/t/indexing-shard-failure/7178 "2012-03-30T13:04:48Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Adam\_Estrada](https://discuss.elastic.co/u/Adam_Estrada)
#### Post date: [March 30, 2012, 1:04pm UTC](https://discuss.elastic.co/t/indexing-shard-failure/7178/1 "2012-03-30T13:04:48Z")
All,
There are a few other posts related to this same thing but I still can't
get my index to recover correctly. I flushed and refreshed the index and
deleted the translog from shard #3. Still no dice. I am running ES 0.18.7
on Windows Server 2008. Any thoughts on how to recover the data I've lost?
[2012-03-30 08:41:21,334][WARN][indices.memory]
> [Geoglobaldomination
> ] failed to set shard [tweets][3] index buffer to [10.1mb]
> [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> [Geoglobaldomination
> ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg], [P],
> s[INI
> TIALIZING], reason [Failed to start shard, message
> [IndexShardGatewayRecoveryExc
> eption[[tweets][3] failed recovery]; nested:
> EngineCreationFailureException[[twe
> ets][3] Failed to open reader on writer]; nested:
> FileNotFoundException[\_1orh.pr
> x]; ]]
> [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> [Geoglobaldomination
> ] received shard failed for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> [P], s[IN
> ITIALIZING], reason [Failed to start shard, message
> [IndexShardGatewayRecoveryEx
> ception[[tweets][3] failed recovery]; nested:
> EngineCreationFailureException[[tw
> eets][3] Failed to open reader on writer]; nested:
> FileNotFoundException[\_1orh.p
> rx]; ]]
> [2012-03-30 08:41:24,881][WARN][indices.cluster]
> [Geoglobaldomination
> ] [tweets][3] failed to start shard
> org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> [tweets][3]
> failed recovery
> at
> org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> hardGatewayService.java:229)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source
> )
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: org.elasticsearch.index.engine.EngineCreationFailureException:
> [tweet
> s][3] Failed to open reader on writer
> at
> org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> va:284)
> at
> org.elasticsearch.index.shard.service.InternalIndexShard.performRecov
> eryPrepareForTranslog(InternalIndexShard.java:535)
> at
> org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(
> LocalIndexShardGateway.java:164)
> at
> org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> hardGatewayService.java:179)
> ... 3 more
> Caused by: java.io.FileNotFoundException: \_1orh.prx
> at
> org.elasticsearch.index.store.Store$StoreDirectory.openInput(Store.ja
> va:450)
> at
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.
> java:89)
> at
> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
> at
> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:7
> 05)
> at
> org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(Index
> Writer.java:663)
> at
> org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:1
> 57)
> at
> org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirect
> oryReader.java:38)
> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:401)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:345)
> at
> org.elasticsearch.index.engine.robin.RobinEngine.buildNrtResource(Rob
> inEngine.java:1365)
> at
> org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> va:268)
> ... 6 more
> [2012-03-30 08:41:24,881][WARN][cluster.action.shard]
> [Geoglobaldomination
> ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg], [P],
> s[INI
> TIALIZING], reason [Failed to start shard, message
> [IndexShardGatewayRecoveryExc
> eption[[tweets][3] failed recovery]; nested:
> EngineCreationFailureException[[twe
> ets][3] Failed to open reader on writer]; nested:
> FileNotFoundException[\_1orh.pr
> x]; ]]
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [March 31, 2012, 8:50pm UTC](https://discuss.elastic.co/t/indexing-shard-failure/7178/2 "2012-03-31T20:50:07Z")
It seems like its missing a specific file for the shard index, how many
nodes do you have in the cluster?
On Fri, Mar 30, 2012 at 4:04 PM, Adam Estrada [estrada.adam@gmail.com](mailto:estrada.adam@gmail.com)wrote:
> All,
>
> There are a few other posts related to this same thing but I still can't
> get my index to recover correctly. I flushed and refreshed the index and
> deleted the translog from shard #3. Still no dice. I am running ES 0.18.7
> on Windows Server 2008. Any thoughts on how to recover the data I've lost?
>
> [2012-03-30 08:41:21,334][WARN][indices.memory]
>
> > [Geoglobaldomination
> > ] failed to set shard [tweets][3] index buffer to [10.1mb]
> > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > [Geoglobaldomination
> > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > [P], s[INI
> > TIALIZING], reason [Failed to start shard, message
> > [IndexShardGatewayRecoveryExc
> > eption[[tweets][3] failed recovery]; nested:
> > EngineCreationFailureException[[twe
> > ets][3] Failed to open reader on writer]; nested: FileNotFoundException[\_
> > 1orh.pr
> > x]; ]]
> > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > [Geoglobaldomination
> > ] received shard failed for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > [P], s[IN
> > ITIALIZING], reason [Failed to start shard, message
> > [IndexShardGatewayRecoveryEx
> > ception[[tweets][3] failed recovery]; nested:
> > EngineCreationFailureException[[tw
> > eets][3] Failed to open reader on writer]; nested:
> > FileNotFoundException[\_1orh.p
> > rx]; ]]
> > [2012-03-30 08:41:24,881][WARN][indices.cluster]
> > [Geoglobaldomination
> > ] [tweets][3] failed to start shard
> > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> > [tweets][3]
> > failed recovery
> > at
> > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > hardGatewayService.java:229)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> > Source
> > )
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> > Source)
> > at java.lang.Thread.run(Unknown Source)
> > Caused by: org.elasticsearch.index.engine.EngineCreationFailureException:
> > [tweet
> > s][3] Failed to open reader on writer
> > at
> > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > va:284)
> > at
> > org.elasticsearch.index.shard.service.InternalIndexShard.performRecov
> > eryPrepareForTranslog(InternalIndexShard.java:535)
> > at
> > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(
> > LocalIndexShardGateway.java:164)
> > at
> > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > hardGatewayService.java:179)
> > ... 3 more
> > Caused by: java.io.FileNotFoundException: _1orh.prx
> > at
> > org.elasticsearch.index.store.Store$StoreDirectory.openInput(Store.ja
> > va:450)
> > at
> > org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.
> > java:89)
> > at
> > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
> > at
> > org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:7
> > 05)
> > at
> > org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(Index
> > Writer.java:663)
> > at
> > org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:1
> > 57)
> > at
> > org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirect
> > oryReader.java:38)
> > at
> > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
> > at
> > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:401)
> > at org.apache.lucene.index.IndexReader.open(IndexReader.java:345)
> > at
> > org.elasticsearch.index.engine.robin.RobinEngine.buildNrtResource(Rob
> > inEngine.java:1365)
> > at
> > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > va:268)
> > ... 6 more
> > [2012-03-30 08:41:24,881][WARN][cluster.action.shard]
> > [Geoglobaldomination
> > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > [P], s[INI
> > TIALIZING], reason [Failed to start shard, message
> > [IndexShardGatewayRecoveryExc
> > eption[[tweets][3] failed recovery]; nested:
> > EngineCreationFailureException[[twe
> > ets][3] Failed to open reader on writer]; nested: FileNotFoundException[_
> > 1orh.pr
> > x]; ]]
---
### Author:  [@Adam\_Estrada](https://discuss.elastic.co/u/Adam_Estrada)
#### Post date: [April 1, 2012, 12:20am UTC](https://discuss.elastic.co/t/indexing-shard-failure/7178/3 "2012-04-01T00:20:55Z")
Shay,
This was running on a single node.
Adam
On Mar 31, 2012 8:50 PM, "Shay Banon" [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> It seems like its missing a specific file for the shard index, how many
> nodes do you have in the cluster?
>
> On Fri, Mar 30, 2012 at 4:04 PM, Adam Estrada [estrada.adam@gmail.com](mailto:estrada.adam@gmail.com)wrote:
>
> > All,
> >
> > There are a few other posts related to this same thing but I still can't
> > get my index to recover correctly. I flushed and refreshed the index and
> > deleted the translog from shard #3. Still no dice. I am running ES 0.18.7
> > on Windows Server 2008. Any thoughts on how to recover the data I've lost?
> >
> > [2012-03-30 08:41:21,334][WARN][indices.memory]
> >
> > > [Geoglobaldomination
> > > ] failed to set shard [tweets][3] index buffer to [10.1mb]
> > > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > > [Geoglobaldomination
> > > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > [P], s[INI
> > > TIALIZING], reason [Failed to start shard, message
> > > [IndexShardGatewayRecoveryExc
> > > eption[[tweets][3] failed recovery]; nested:
> > > EngineCreationFailureException[[twe
> > > ets][3] Failed to open reader on writer]; nested: FileNotFoundException[\_
> > > 1orh.pr
> > > x]; ]]
> > > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > > [Geoglobaldomination
> > > ] received shard failed for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > [P], s[IN
> > > ITIALIZING], reason [Failed to start shard, message
> > > [IndexShardGatewayRecoveryEx
> > > ception[[tweets][3] failed recovery]; nested:
> > > EngineCreationFailureException[[tw
> > > eets][3] Failed to open reader on writer]; nested:
> > > FileNotFoundException[\_1orh.p
> > > rx]; ]]
> > > [2012-03-30 08:41:24,881][WARN][indices.cluster]
> > > [Geoglobaldomination
> > > ] [tweets][3] failed to start shard
> > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> > > [tweets][3]
> > > failed recovery
> > > at
> > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > > hardGatewayService.java:229)
> > > at
> > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source
> > > )
> > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> > > Source)
> > > at java.lang.Thread.run(Unknown Source)
> > > Caused by:
> > > org.elasticsearch.index.engine.EngineCreationFailureException: [tweet
> > > s][3] Failed to open reader on writer
> > > at
> > > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > > va:284)
> > > at
> > > org.elasticsearch.index.shard.service.InternalIndexShard.performRecov
> > > eryPrepareForTranslog(InternalIndexShard.java:535)
> > > at
> > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(
> > > LocalIndexShardGateway.java:164)
> > > at
> > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > > hardGatewayService.java:179)
> > > ... 3 more
> > > Caused by: java.io.FileNotFoundException: _1orh.prx
> > > at
> > > org.elasticsearch.index.store.Store$StoreDirectory.openInput(Store.ja
> > > va:450)
> > > at
> > > org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.
> > > java:89)
> > > at
> > > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
> > > at
> > > org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:7
> > > 05)
> > > at
> > > org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(Index
> > > Writer.java:663)
> > > at
> > > org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:1
> > > 57)
> > > at
> > > org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirect
> > > oryReader.java:38)
> > > at
> > > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
> > > at
> > > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:401)
> > > at org.apache.lucene.index.IndexReader.open(IndexReader.java:345)
> > > at
> > > org.elasticsearch.index.engine.robin.RobinEngine.buildNrtResource(Rob
> > > inEngine.java:1365)
> > > at
> > > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > > va:268)
> > > ... 6 more
> > > [2012-03-30 08:41:24,881][WARN][cluster.action.shard]
> > > [Geoglobaldomination
> > > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > [P], s[INI
> > > TIALIZING], reason [Failed to start shard, message
> > > [IndexShardGatewayRecoveryExc
> > > eption[[tweets][3] failed recovery]; nested:
> > > EngineCreationFailureException[[twe
> > > ets][3] Failed to open reader on writer]; nested: FileNotFoundException[_
> > > 1orh.pr
> > > x]; ]]
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [April 3, 2012, 2:04pm UTC](https://discuss.elastic.co/t/indexing-shard-failure/7178/4 "2012-04-03T14:04:34Z")
I see, so it has no replica to recover from. The strange this is that this
single file that is missing will not get deleted by anything in
elasticsearch or Lucene, its either all gets deleted (and they you get a
different message) or nothing... . Maybe it somehow got deleted by mistake?
Which version are you using?
On Sun, Apr 1, 2012 at 3:20 AM, Adam Estrada [estrada.adam@gmail.com](mailto:estrada.adam@gmail.com) wrote:
> Shay,
>
> This was running on a single node.
>
> Adam
> On Mar 31, 2012 8:50 PM, "Shay Banon" [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
>
> > It seems like its missing a specific file for the shard index, how many
> > nodes do you have in the cluster?
> >
> > On Fri, Mar 30, 2012 at 4:04 PM, Adam Estrada [estrada.adam@gmail.com](mailto:estrada.adam@gmail.com)wrote:
> >
> > > All,
> > >
> > > There are a few other posts related to this same thing but I still can't
> > > get my index to recover correctly. I flushed and refreshed the index and
> > > deleted the translog from shard #3. Still no dice. I am running ES 0.18.7
> > > on Windows Server 2008. Any thoughts on how to recover the data I've lost?
> > >
> > > [2012-03-30 08:41:21,334][WARN][indices.memory]
> > >
> > > > [Geoglobaldomination
> > > > ] failed to set shard [tweets][3] index buffer to [10.1mb]
> > > > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > > > [Geoglobaldomination
> > > > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > > [P], s[INI
> > > > TIALIZING], reason [Failed to start shard, message
> > > > [IndexShardGatewayRecoveryExc
> > > > eption[[tweets][3] failed recovery]; nested:
> > > > EngineCreationFailureException[[twe
> > > > ets][3] Failed to open reader on writer]; nested:
> > > > FileNotFoundException[\_1orh.pr
> > > > x]; ]]
> > > > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > > > [Geoglobaldomination
> > > > ] received shard failed for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > > [P], s[IN
> > > > ITIALIZING], reason [Failed to start shard, message
> > > > [IndexShardGatewayRecoveryEx
> > > > ception[[tweets][3] failed recovery]; nested:
> > > > EngineCreationFailureException[[tw
> > > > eets][3] Failed to open reader on writer]; nested:
> > > > FileNotFoundException[\_1orh.p
> > > > rx]; ]]
> > > > [2012-03-30 08:41:24,881][WARN][indices.cluster]
> > > > [Geoglobaldomination
> > > > ] [tweets][3] failed to start shard
> > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> > > > [tweets][3]
> > > > failed recovery
> > > > at
> > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > > > hardGatewayService.java:229)
> > > > at
> > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source
> > > > )
> > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> > > > Source)
> > > > at java.lang.Thread.run(Unknown Source)
> > > > Caused by:
> > > > org.elasticsearch.index.engine.EngineCreationFailureException: [tweet
> > > > s][3] Failed to open reader on writer
> > > > at
> > > > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > > > va:284)
> > > > at
> > > > org.elasticsearch.index.shard.service.InternalIndexShard.performRecov
> > > > eryPrepareForTranslog(InternalIndexShard.java:535)
> > > > at
> > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(
> > > > LocalIndexShardGateway.java:164)
> > > > at
> > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > > > hardGatewayService.java:179)
> > > > ... 3 more
> > > > Caused by: java.io.FileNotFoundException: \_1orh.prx
> > > > at
> > > > org.elasticsearch.index.store.Store$StoreDirectory.openInput(Store.ja
> > > > va:450)
> > > > at
> > > > org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.
> > > > java:89)
> > > > at
> > > > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
> > > > at
> > > > org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:7
> > > > 05)
> > > > at
> > > > org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(Index
> > > > Writer.java:663)
> > > > at
> > > > org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:1
> > > > 57)
> > > > at
> > > > org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirect
> > > > oryReader.java:38)
> > > > at
> > > > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
> > > > at
> > > > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:401)
> > > > at
> > > > org.apache.lucene.index.IndexReader.open(IndexReader.java:345)
> > > > at
> > > > org.elasticsearch.index.engine.robin.RobinEngine.buildNrtResource(Rob
> > > > inEngine.java:1365)
> > > > at
> > > > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > > > va:268)
> > > > ... 6 more
> > > > [2012-03-30 08:41:24,881][WARN][cluster.action.shard]
> > > > [Geoglobaldomination
> > > > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > > [P], s[INI
> > > > TIALIZING], reason [Failed to start shard, message
> > > > [IndexShardGatewayRecoveryExc
> > > > eption[[tweets][3] failed recovery]; nested:
> > > > EngineCreationFailureException[[twe
> > > > ets][3] Failed to open reader on writer]; nested:
> > > > FileNotFoundException[\_1orh.pr
> > > > x]; ]]
---
### Author:  [@Adam\_Estrada](https://discuss.elastic.co/u/Adam_Estrada)
#### Post date: [April 4, 2012, 9:47pm UTC](https://discuss.elastic.co/t/indexing-shard-failure/7178/5 "2012-04-04T21:47:56Z")
Shay,
I was using 0.87.1 I think but have since switched to the latest version of
ES. I did find it odd that the single index file was missing. I searched
through the recycle bin and all that and it just doesn't exist any more.
Not worry...Data was lost but its not the end of the world 😉
Adam
On Tue, Apr 3, 2012 at 10:04 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> I see, so it has no replica to recover from. The strange this is that this
> single file that is missing will not get deleted by anything in
> elasticsearch or Lucene, its either all gets deleted (and they you get a
> different message) or nothing... . Maybe it somehow got deleted by mistake?
> Which version are you using?
>
> On Sun, Apr 1, 2012 at 3:20 AM, Adam Estrada [estrada.adam@gmail.com](mailto:estrada.adam@gmail.com)wrote:
>
> > Shay,
> >
> > This was running on a single node.
> >
> > Adam
> > On Mar 31, 2012 8:50 PM, "Shay Banon" [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> >
> > > It seems like its missing a specific file for the shard index, how many
> > > nodes do you have in the cluster?
> > >
> > > On Fri, Mar 30, 2012 at 4:04 PM, Adam Estrada [estrada.adam@gmail.com](mailto:estrada.adam@gmail.com)wrote:
> > >
> > > > All,
> > > >
> > > > There are a few other posts related to this same thing but I still
> > > > can't get my index to recover correctly. I flushed and refreshed the index
> > > > and deleted the translog from shard #3. Still no dice. I am running ES
> > > > 0.18.7 on Windows Server 2008. Any thoughts on how to recover the data I've
> > > > lost?
> > > >
> > > > [2012-03-30 08:41:21,334][WARN][indices.memory]
> > > >
> > > > > [Geoglobaldomination
> > > > > ] failed to set shard [tweets][3] index buffer to [10.1mb]
> > > > > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > > > > [Geoglobaldomination
> > > > > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > > > [P], s[INI
> > > > > TIALIZING], reason [Failed to start shard, message
> > > > > [IndexShardGatewayRecoveryExc
> > > > > eption[[tweets][3] failed recovery]; nested:
> > > > > EngineCreationFailureException[[twe
> > > > > ets][3] Failed to open reader on writer]; nested:
> > > > > FileNotFoundException[\_1orh.pr
> > > > > x]; ]]
> > > > > [2012-03-30 08:41:21,741][WARN][cluster.action.shard]
> > > > > [Geoglobaldomination
> > > > > ] received shard failed for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > > > [P], s[IN
> > > > > ITIALIZING], reason [Failed to start shard, message
> > > > > [IndexShardGatewayRecoveryEx
> > > > > ception[[tweets][3] failed recovery]; nested:
> > > > > EngineCreationFailureException[[tw
> > > > > eets][3] Failed to open reader on writer]; nested:
> > > > > FileNotFoundException[\_1orh.p
> > > > > rx]; ]]
> > > > > [2012-03-30 08:41:24,881][WARN][indices.cluster]
> > > > > [Geoglobaldomination
> > > > > ] [tweets][3] failed to start shard
> > > > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
> > > > > [tweets][3]
> > > > > failed recovery
> > > > > at
> > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > > > > hardGatewayService.java:229)
> > > > > at
> > > > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source
> > > > > )
> > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> > > > > Source)
> > > > > at java.lang.Thread.run(Unknown Source)
> > > > > Caused by:
> > > > > org.elasticsearch.index.engine.EngineCreationFailureException: [tweet
> > > > > s][3] Failed to open reader on writer
> > > > > at
> > > > > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > > > > va:284)
> > > > > at
> > > > > org.elasticsearch.index.shard.service.InternalIndexShard.performRecov
> > > > > eryPrepareForTranslog(InternalIndexShard.java:535)
> > > > > at
> > > > > org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(
> > > > > LocalIndexShardGateway.java:164)
> > > > > at
> > > > > org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexS
> > > > > hardGatewayService.java:179)
> > > > > ... 3 more
> > > > > Caused by: java.io.FileNotFoundException: \_1orh.prx
> > > > > at
> > > > > org.elasticsearch.index.store.Store$StoreDirectory.openInput(Store.ja
> > > > > va:450)
> > > > > at
> > > > > org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.
> > > > > java:89)
> > > > > at
> > > > > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
> > > > > at
> > > > > org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:7
> > > > > 05)
> > > > > at
> > > > > org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(Index
> > > > > Writer.java:663)
> > > > > at
> > > > > org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:1
> > > > > 57)
> > > > > at
> > > > > org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirect
> > > > > oryReader.java:38)
> > > > > at
> > > > > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
> > > > > at
> > > > > org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:401)
> > > > > at
> > > > > org.apache.lucene.index.IndexReader.open(IndexReader.java:345)
> > > > > at
> > > > > org.elasticsearch.index.engine.robin.RobinEngine.buildNrtResource(Rob
> > > > > inEngine.java:1365)
> > > > > at
> > > > > org.elasticsearch.index.engine.robin.RobinEngine.start(RobinEngine.ja
> > > > > va:268)
> > > > > ... 6 more
> > > > > [2012-03-30 08:41:24,881][WARN][cluster.action.shard]
> > > > > [Geoglobaldomination
> > > > > ] sending failed shard for [tweets][3], node[YsQ2QjHdTcG184MVUcXUVg],
> > > > > [P], s[INI
> > > > > TIALIZING], reason [Failed to start shard, message
> > > > > [IndexShardGatewayRecoveryExc
> > > > > eption[[tweets][3] failed recovery]; nested:
> > > > > EngineCreationFailureException[[twe
> > > > > ets][3] Failed to open reader on writer]; nested:
> > > > > FileNotFoundException[\_1orh.pr
> > > > > x]; ]]
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:33am UTC](https://discuss.elastic.co/t/indexing-shard-failure/7178/6 "2017-07-06T03:33:32Z")