No commit point data in gateway

Elastic Stack Elasticsearch
1

I'm seeing this error message appear in my startup logs, which seems to be
preventing the ES cluster from reaching YELLOW/GREEN state.

16:42:27,642 WARN thread-1 arch.indices.cluster: 90 - [Supernalia]
[ppkc][1] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException:
[ppkc][1] No commit point data is available in gateway
at
org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.recover(BlobStoreIndexShardGateway.java:423)
at
org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:144)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

In this gist is the full log output along with my ES configuration params.
Running 0.16.0 on EC2, clustering not working yet.

All I have done is redeploy a new version of my webapp to a running app
server instance. The EBS block is the same as the previous and ES is using
local indicies. I'm not sure what happended to the so-called commit point
data, but I have seen this error frequently.

2

The way that the s3 gateway works is that it writes a commit point file listing all the files relevant for that shard after it has written all the files. And then, when recovering, it reads the commit point file, and recover all the data. It seems like some files are missing for that shard at s3. Not really sure how it got to that state, but it means that the relevant shard data is basically lost.

Is this something that you can recreate consistently? I would like to try and recreate it.
On Friday, May 20, 2011 at 7:49 PM, James Cook wrote:

I'm seeing this error message appear in my startup logs, which seems to be preventing the ES cluster from reaching YELLOW/GREEN state.

16:42:27,642 WARN thread-1 arch.indices.cluster: 90 - [Supernalia] [ppkc][1] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [ppkc][1] No commit point data is available in gateway
at org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.recover(BlobStoreIndexShardGateway.java:423)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:144)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

In this gist is the full log output along with my ES configuration params. Running 0.16.0 on EC2, clustering not working yet.
gist:983300 · GitHub

All I have done is redeploy a new version of my webapp to a running app server instance. The EBS block is the same as the previous and ES is using local indicies. I'm not sure what happended to the so-called commit point data, but I have seen this error frequently.