# error trying to access river - IndexShardGatewayRecoveryException **URL:** \ **Category:** Elasticsearch\ **Created:** [August 11, 2012, 10:24pm UTC](https://discuss.elastic.co/t/error-trying-to-access-river-indexshardgatewayrecoveryexception/8721 "2012-08-11T22:24:06Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Warner\_Onstine](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warner_onstine/32/1086_2.png) [@Warner\_Onstine](https://discuss.elastic.co/u/Warner_Onstine)\ **Post date:** [August 11, 2012, 10:24pm UTC](https://discuss.elastic.co/t/error-trying-to-access-river-indexshardgatewayrecoveryexception/8721/1 "2012-08-11T22:24:06Z")
So, I my server ran out of space (which is possibly what started this problem), but now I can't get my couchdb river going again. I've deleted my river and when I try and recreate it I get this error: {"error":"UnavailableShardsException[[\_river][0] [2] shardIt, [0] active : Timeout waiting for [1m], request: index {[\_river][syncrswim][\_meta], source[{"type" : "couchdb","couchdb" : {"host" : "localhost","port" : 5984,"db" :"syncrswim","filter" : null},"index" : {"index" : "syncrswim", "type" : "syncrswim", "bulk\_size" : "100", "bulk\_timeout" : "10ms" }}]}]","status":503} before it was removed I was seeing this in the log [2012-08-11 14:45:37,883][WARN][indices.cluster] [Golubev, Mikula] [\_river][0] failed to start shard org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [\_river][0] failed to recover shard at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:216) at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0 at java.lang.String.charAt(String.java:694) at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:182) at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:174) at org.elasticsearch.index.mapper.MapperService.documentMapperWithAutoCreate(MapperService.java:300) at org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:310) at org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryOperation(InternalIndexShard.java:624) at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:211) ... 4 more [2012-08-11 14:45:37,890][WARN][cluster.action.shard] [Golubev, Mikula] sending failed shard for [\_river][0], node[I0pOvJWbQoSVJDqQb3ZZ2A], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[\_river][0] failed to recover shard]; nested: StringIndexOutOfBoundsException[String index out of range: 0]; ]] [2012-08-11 14:45:37,890][WARN][cluster.action.shard] [Golubev, Mikula] received shard failed for [\_river][0], node[I0pOvJWbQoSVJDqQb3ZZ2A], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[\_river][0] failed to recover shard]; nested: StringIndexOutOfBoundsException[String index out of range: 0]; ]] Any thoughts on how to recover from this? I'm still in development mode so if need be I can wipe out my data, just not sure how to do this in a safe way. -warner -- ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [August 13, 2012, 10:23am UTC](https://discuss.elastic.co/t/error-trying-to-access-river-indexshardgatewayrecoveryexception/8721/2 "2012-08-13T10:23:07Z")
This will be fixed in 0.19.9 where there is better handling of corrupt translog. Now, you can simply delete the "data" location of ES if you want to start fresh, or, in case you would have run this in production, and you had replicas allocated, those replicas would have been used for recovery automatically. On Aug 12, 2012, at 12:24 AM, Warner Onstine [warnero@gmail.com](mailto:warnero@gmail.com) wrote: > So, I my server ran out of space (which is possibly what started this > problem), but now I can't get my couchdb river going again. > > I've deleted my river and when I try and recreate it I get this error: > {"error":"UnavailableShardsException[[\_river][0] [2] shardIt, [0] > active : Timeout waiting for [1m], request: index > {[\_river][syncrswim][\_meta], source[{"type" : > "couchdb","couchdb" : {"host" : "localhost","port" : > 5984,"db" :"syncrswim","filter" : null},"index" : {"index" : > "syncrswim", "type" : "syncrswim", "bulk\_size" : > "100", "bulk\_timeout" : "10ms" }}]}]","status":503} > > before it was removed I was seeing this in the log > > [2012-08-11 14:45:37,883][WARN][indices.cluster] [Golubev, > Mikula] [\_river][0] failed to start shard > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > [\_river][0] failed to recover shard > at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:216) > at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:177) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > Caused by: java.lang.StringIndexOutOfBoundsException: String index out > of range: 0 > at java.lang.String.charAt(String.java:694) > at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:182) > at org.elasticsearch.index.mapper.MapperService.add(MapperService.java:174) > at org.elasticsearch.index.mapper.MapperService.documentMapperWithAutoCreate(MapperService.java:300) > at org.elasticsearch.index.shard.service.InternalIndexShard.prepareIndex(InternalIndexShard.java:310) > at org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryOperation(InternalIndexShard.java:624) > at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:211) > ... 4 more > [2012-08-11 14:45:37,890][WARN][cluster.action.shard] [Golubev, > Mikula] sending failed shard for [\_river][0], > node[I0pOvJWbQoSVJDqQb3ZZ2A], [P], s[INITIALIZING], reason [Failed to > start shard, message [IndexShardGatewayRecoveryException[[\_river][0] > failed to recover shard]; nested: > StringIndexOutOfBoundsException[String index out of range: 0]; ]] > [2012-08-11 14:45:37,890][WARN][cluster.action.shard] [Golubev, > Mikula] received shard failed for [\_river][0], > node[I0pOvJWbQoSVJDqQb3ZZ2A], [P], s[INITIALIZING], reason [Failed to > start shard, message [IndexShardGatewayRecoveryException[[\_river][0] > failed to recover shard]; nested: > StringIndexOutOfBoundsException[String index out of range: 0]; ]] > > Any thoughts on how to recover from this? I'm still in development > mode so if need be I can wipe out my data, just not sure how to do > this in a safe way. > > -warner > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:16am UTC](https://discuss.elastic.co/t/error-trying-to-access-river-indexshardgatewayrecoveryexception/8721/3 "2017-07-06T03:16:35Z")