Received shard failed on twitter_river

Anurag_Phadke · December 20, 2011, 12:36am

Hello,
We are using ElasticSearch 0.17.4 and had twitter_river running for
past 6 months which resulted in OOME error.
I ended up deleting the twitter plugin folder for jars and relevant
river_twitter index
/data01/elasticsearch/data/buildbot/nodes/*/indices/twitter_river

However, even after doing the above and restarting the cluster, I get
the following in logs:

[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,115][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,192][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

any idea what might be happening out here?

-anurag

--
Blog: http://anurag.co
Twitter: @anuragphadke (https://twitter.com/#!/anuragphadke)

kimchy · December 20, 2011, 10:01am

Deleting the "folders" will not cause the index metadata to be deleted,
just send a delete index request against that index, it will be removed.

On Tue, Dec 20, 2011 at 2:36 AM, Anurag anurag.phadke@gmail.com wrote:

Hello,
We are using Elasticsearch 0.17.4 and had twitter_river running for
past 6 months which resulted in OOME error.
I ended up deleting the twitter plugin folder for jars and relevant
river_twitter index
/data01/elasticsearch/data/buildbot/nodes/*/indices/twitter_river

However, even after doing the above and restarting the cluster, I get
the following in logs:

[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,115][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,192][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

any idea what might be happening out here?

-anurag

--
Blog: http://anurag.co
Twitter: @anuragphadke (https://twitter.com/#!/anuragphadke)

Anurag_Phadke · December 20, 2011, 1:49pm

I tried sending it few times, returns back with a {ok} response, but the index doesn't get deleted. Is it because the cluster status is red?

On Dec 20, 2011, at 2:01 AM, Shay Banon kimchy@gmail.com wrote:

Deleting the "folders" will not cause the index metadata to be deleted, just send a delete index request against that index, it will be removed.

On Tue, Dec 20, 2011 at 2:36 AM, Anurag anurag.phadke@gmail.com wrote:
Hello,
We are using Elasticsearch 0.17.4 and had twitter_river running for
past 6 months which resulted in OOME error.
I ended up deleting the twitter plugin folder for jars and relevant
river_twitter index
/data01/elasticsearch/data/buildbot/nodes/*/indices/twitter_river

However, even after doing the above and restarting the cluster, I get
the following in logs:

[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,115][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,192][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

any idea what might be happening out here?

-anurag

--
Blog: http://anurag.co
Twitter: @anuragphadke (https://twitter.com/#!/anuragphadke)

kimchy · December 20, 2011, 2:55pm

No, delete index will be allowed to get in. Have you waited a bit and the
failures still continue?

On Tue, Dec 20, 2011 at 3:49 PM, Anurag Phadke anurag.phadke@gmail.comwrote:

I tried sending it few times, returns back with a {ok} response, but the
index doesn't get deleted. Is it because the cluster status is red?

On Dec 20, 2011, at 2:01 AM, Shay Banon kimchy@gmail.com wrote:

Deleting the "folders" will not cause the index metadata to be deleted,
just send a delete index request against that index, it will be removed.

On Tue, Dec 20, 2011 at 2:36 AM, Anurag < anurag.phadke@gmail.com
anurag.phadke@gmail.com> wrote:

Hello,
We are using Elasticsearch 0.17.4 and had twitter_river running for
past 6 months which resulted in OOME error.
I ended up deleting the twitter plugin folder for jars and relevant
river_twitter index
/data01/elasticsearch/data/buildbot/nodes/*/indices/twitter_river

However, even after doing the above and restarting the cluster, I get
the following in logs:

[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,115][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,192][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

any idea what might be happening out here?

-anurag

--
Blog: http://anurag.co http://anurag.co
Twitter: @anuragphadke ( https://twitter.com/#!/anuragphadke
https://twitter.com/#!/anuragphadke)

Anurag_Phadke · December 20, 2011, 3:18pm

I waited for about 15m. Lemme retry again and wait for a longer time.

On Dec 20, 2011, at 6:55 AM, Shay Banon kimchy@gmail.com wrote:

No, delete index will be allowed to get in. Have you waited a bit and the failures still continue?

On Tue, Dec 20, 2011 at 3:49 PM, Anurag Phadke anurag.phadke@gmail.com wrote:
I tried sending it few times, returns back with a {ok} response, but the index doesn't get deleted. Is it because the cluster status is red?

On Dec 20, 2011, at 2:01 AM, Shay Banon kimchy@gmail.com wrote:

Deleting the "folders" will not cause the index metadata to be deleted, just send a delete index request against that index, it will be removed.

On Tue, Dec 20, 2011 at 2:36 AM, Anurag anurag.phadke@gmail.com wrote:
Hello,
We are using Elasticsearch 0.17.4 and had twitter_river running for
past 6 months which resulted in OOME error.
I ended up deleting the twitter plugin folder for jars and relevant
river_twitter index
/data01/elasticsearch/data/buildbot/nodes/*/indices/twitter_river

However, even after doing the above and restarting the cluster, I get
the following in logs:

[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,115][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,192][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

any idea what might be happening out here?

-anurag

--
Blog: http://anurag.co
Twitter: @anuragphadke (https://twitter.com/#!/anuragphadke)

kimchy · December 20, 2011, 3:34pm

Just make sure you delete the relevant index.

On Tue, Dec 20, 2011 at 5:18 PM, Anurag Phadke anurag.phadke@gmail.comwrote:

I waited for about 15m. Lemme retry again and wait for a longer time.

On Dec 20, 2011, at 6:55 AM, Shay Banon kimchy@gmail.com wrote:

No, delete index will be allowed to get in. Have you waited a bit and the
failures still continue?

On Tue, Dec 20, 2011 at 3:49 PM, Anurag Phadke < anurag.phadke@gmail.com
anurag.phadke@gmail.com> wrote:

I tried sending it few times, returns back with a {ok} response, but the
index doesn't get deleted. Is it because the cluster status is red?

On Dec 20, 2011, at 2:01 AM, Shay Banon < kimchy@gmail.com
kimchy@gmail.com> wrote:

Deleting the "folders" will not cause the index metadata to be deleted,
just send a delete index request against that index, it will be removed.

On Tue, Dec 20, 2011 at 2:36 AM, Anurag < anurag.phadke@gmail.com anurag.phadke@gmail.com
anurag.phadke@gmail.com> wrote:

Hello,
We are using Elasticsearch 0.17.4 and had twitter_river running for
past 6 months which resulted in OOME error.
I ended up deleting the twitter plugin folder for jars and relevant
river_twitter index
/data01/elasticsearch/data/buildbot/nodes/*/indices/twitter_river

However, even after doing the above and restarting the cluster, I get
the following in logs:

[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,086][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,115][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] sending failed shard for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,117][WARN ][cluster.action.shard ] [Will o'
the Wisp] received shard failed for [twitter_river][3],
node[KcI_F3nnSWyhvm_-McxDTg], [P], s[INITIALIZING], reason [Failed to
start shard, message
[IndexShardGatewayRecoveryException[[twitter_river][3] shard allocated
for local recovery (post api), should exists, but doesn't]]]
[2011-12-19 16:32:34,192][WARN ][indices.cluster ] [Will o'
the Wisp] [twitter_river][3] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[twitter_river][3] shard allocated for local recovery (post api),
should exists, but doesn't
at
org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:99)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:179)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

any idea what might be happening out here?

-anurag

--
Blog: http://anurag.co http://anurag.co http://anurag.co
Twitter: @anuragphadke ( https://twitter.com/#!/anuragphadke https://twitter.com/#!/anuragphadke
https://twitter.com/#!/anuragphadke)