Hello,
We are running Elasticsearch 2.3.5.
About a month ago and a half ago, we optimized all of our indexes. E.g.:
POST /w-indexname-2016.07.21_v1.4/_optimize?max_num_segments=1
Since then, when we run our daily snapshop, we get this log entry for many of our indexes and shards:
[w-indexname-2016.07.21_v1.4][[w-indexname-2016.07.21_v1.4][2]] IndexShardSnapshotFailedException[class org.apache.lucene.store.BufferedChecksumIndexInput cannot seek backwards (pos=-16 getFilePointer()=0)]; nested: IllegalStateException[class org.apache.lucene.store.BufferedChecksumIndexInput cannot seek backwards (pos=-16 getFilePointer()=0)];
at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:201)
at org.elasticsearch.snapshots.SnapshotShardsService.snapshot(SnapshotShardsService.java:340)
at org.elasticsearch.snapshots.SnapshotShardsService.access$200(SnapshotShardsService.java:76)
at org.elasticsearch.snapshots.SnapshotShardsService$1.doRun(SnapshotShardsService.java:296)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: class org.apache.lucene.store.BufferedChecksumIndexInput cannot seek backwards (pos=-16 getFilePointer()=0)
at org.apache.lucene.store.ChecksumIndexInput.seek(ChecksumIndexInput.java:50)
at org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:450)
at org.elasticsearch.repositories.blobstore.ChecksumBlobStoreFormat.readBlob(ChecksumBlobStoreFormat.java:97)
at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$Context.buildBlobStoreIndexShardSnapshots(BlobStoreIndexShardRepository.java:481)
at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext.snapshot(BlobStoreIndexShardRepository.java:537)
at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:191)
... 7 more
Any thoughts?