Hello! i have snapshot repository which is located on NFS share, and i'm try to register repo from my local ES instance and got strange error
curl -X PUT "192.168.2.11:9200/_snapshot/delete_after_read?pretty" -H 'Content-Type: application/json' -d'
{
"type": "url",
"settings": {
"url": "file:/opt/elasticsearch-backup/elastique-backup"
}
}
'
{
"error" : {
"root_cause" : [
{
"type" : "array_index_out_of_bounds_exception",
"reason" : "Index 0 out of bounds for length 0"
}
],
"type" : "array_index_out_of_bounds_exception",
"reason" : "Index 0 out of bounds for length 0"
},
"status" : 500
}
why this error occured?
spinscale
(Alexander Reelsen)
September 6, 2021, 2:30pm
2
Can you share the output of that call when adding the parameter error_trace=true
and share the whole output in a gist?
That said, how about using the fs
repository instead? Alternatively you may want to try file:///opt/...
for the url
.
1 Like
warkolm
(Mark Walkom)
September 6, 2021, 11:46pm
3
Shouldn't you be using;
curl -X PUT "host:9200/_snapshot/delete_after_read?pretty" -H 'Content-Type: application/json' -d'
{
"type": "fs",
"settings": {
"location": "/opt/elasticsearch-backup/elastique-backup",
}
}
As per Register a snapshot repository | Elasticsearch Guide [7.14] | Elastic ?
With error trace enabled
curl -X PUT "192.168.2.11:9200/_snapshot/delete_after_read?pretty&error_trace=true" -H 'Content-Type: application/json' -d'
{
"type": "url",
"settings": {
"url": "file:/opt/elasticsearch-backup/elastique-backup"
}
}
'
{
"error" : {
"root_cause" : [
{
"type" : "repository_verification_exception",
"reason" : "[delete_after_read] path is not accessible on master node",
"stack_trace" : "RepositoryVerificationException[[delete_after_read] path is not accessible on master node]; nested: ArrayIndexOutOfBoundsException[Index 0 out of bounds for length 0];\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:1648)\n\tat org.elasticsearch.repositories.RepositoriesService$4.doRun(RepositoriesService.java:392)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\n"
}
],
"type" : "repository_verification_exception",
"reason" : "[delete_after_read] path is not accessible on master node",
"caused_by" : {
"type" : "array_index_out_of_bounds_exception",
"reason" : "Index 0 out of bounds for length 0",
"stack_trace" : "java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:1648)\n\tat org.elasticsearch.repositories.RepositoriesService$4.doRun(RepositoriesService.java:392)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\n"
},
"stack_trace" : "RepositoryVerificationException[[delete_after_read] path is not accessible on master node]; nested: ArrayIndexOutOfBoundsException[Index 0 out of bounds for length 0];\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:1648)\n\tat org.elasticsearch.repositories.RepositoriesService$4.doRun(RepositoriesService.java:392)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\n"
},
"status" : 500
}
{
"type": "fs",
"settings": {
"location": "/opt/elasticsearch-backup/elastique-backup",
"compress": true
}
}
'
{
"error" : {
"root_cause" : [
{
"type" : "parsing_exception",
"reason" : "Failed to parse object: expecting token of type [null] but found [VALUE_NUMBER]",
"line" : 1,
"col" : 9085,
"stack_trace" : "ParsingException[Failed to parse object: expecting token of type [null] but found [VALUE_NUMBER]]\n\tat org.elasticsearch.common.xcontent.XContentParserUtils.parsingException(XContentParserUtils.java:74)\n\tat org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken(XContentParserUtils.java:68)\n\tat org.elasticsearch.repositories.RepositoryData.snapshotsFromXContent(RepositoryData.java:819)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.getRepositoryData(BlobStoreRepository.java:2099)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1906)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\n"
}
],
"type" : "repository_exception",
"reason" : "[delete_after_read] Unexpected exception when loading repository data",
"caused_by" : {
"type" : "parsing_exception",
"reason" : "Failed to parse object: expecting token of type [null] but found [VALUE_NUMBER]",
"line" : 1,
"col" : 9085,
"stack_trace" : "ParsingException[Failed to parse object: expecting token of type [null] but found [VALUE_NUMBER]]\n\tat org.elasticsearch.common.xcontent.XContentParserUtils.parsingException(XContentParserUtils.java:74)\n\tat org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken(XContentParserUtils.java:68)\n\tat org.elasticsearch.repositories.RepositoryData.snapshotsFromXContent(RepositoryData.java:819)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.getRepositoryData(BlobStoreRepository.java:2099)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1906)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\n"
},
"stack_trace" : "RepositoryException[[delete_after_read] Unexpected exception when loading repository data]; nested: ParsingException[Failed to parse object: expecting token of type [null] but found [VALUE_NUMBER]];\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1960)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\nCaused by: ParsingException[Failed to parse object: expecting token of type [null] but found [VALUE_NUMBER]]\n\tat org.elasticsearch.common.xcontent.XContentParserUtils.parsingException(XContentParserUtils.java:74)\n\tat org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken(XContentParserUtils.java:68)\n\tat org.elasticsearch.repositories.RepositoryData.snapshotsFromXContent(RepositoryData.java:819)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.getRepositoryData(BlobStoreRepository.java:2099)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1906)\n\t... 6 more\n"
},
"status" : 500
}
spinscale
(Alexander Reelsen)
September 7, 2021, 7:18am
6
Yeah! it supposed to be readonly, but i have strange errors with registering
And almost same thing when i register another repository ( which located on the same NFS share )
curl -X PUT "192.168.2.11:9200/_snapshot/forteid?pretty&error_trace=true" -H 'Content-Type: application/json' -d'
> {
> "type": "url",
> "settings": {
> "url": "file:/opt/elasticsearch-backup/prod-forteid-efk-7.4.1"
> }
> }
> '
{
"error" : {
"root_cause" : [
{
"type" : "repository_exception",
"reason" : "[forteid] Could not determine repository generation from root blobs",
"stack_trace" : "RepositoryException[[forteid] Could not determine repository generation from root blobs]; nested: ArrayIndexOutOfBoundsException[Index 0 out of bounds for length 0];\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1884)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1881)\n\t... 6 more\n"
}
],
"type" : "repository_exception",
"reason" : "[forteid] Could not determine repository generation from root blobs",
"caused_by" : {
"type" : "array_index_out_of_bounds_exception",
"reason" : "Index 0 out of bounds for length 0",
"stack_trace" : "java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1881)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\n"
},
"stack_trace" : "RepositoryException[[forteid] Could not determine repository generation from root blobs]; nested: ArrayIndexOutOfBoundsException[Index 0 out of bounds for length 0];\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1884)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1881)\n\t... 6 more\n"
},
"status" : 500
}
Guys, any ideas to fix that?
spinscale
(Alexander Reelsen)
September 13, 2021, 2:04pm
10
can you try the file:///
suggestio I mentioned above and tell if it worked?
curl -X PUT "192.168.2.11:9200/_snapshot/delete_after_read?pretty" -H 'Content-Type: application/json' -d'
{
"type": "url",
"settings": {
"file:///": "file:/opt/elasticsearch-backup/elastique-backup"
}
}
'
like this?
spinscale
(Alexander Reelsen)
September 14, 2021, 7:37am
12
No, not in the key, but in the value of that field.
@spinscale same error with file:///
curl -X PUT "192.168.2.11:9200/_snapshot/lento4ka?pretty&error_trace=true" -H 'Content-Type: application/json' -d'
{
"type": "url",
"settings": {
"url": "file:///opt/lenta/efk-anthill-nopci-14.03.2021"
}
}
'
{
"error" : {
"root_cause" : [
{
"type" : "repository_exception",
"reason" : "[lento4ka] Could not determine repository generation from root blobs",
"stack_trace" : "RepositoryException[[lento4ka] Could not determine repository generation from root blobs]; nested: ArrayIndexOutOfBoundsException[Index 0 out of bounds for length 0];\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1884)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1881)\n\t... 6 more\n"
}
],
"type" : "repository_exception",
"reason" : "[lento4ka] Could not determine repository generation from root blobs",
"caused_by" : {
"type" : "array_index_out_of_bounds_exception",
"reason" : "Index 0 out of bounds for length 0",
"stack_trace" : "java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1881)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\n"
},
"stack_trace" : "RepositoryException[[lento4ka] Could not determine repository generation from root blobs]; nested: ArrayIndexOutOfBoundsException[Index 0 out of bounds for length 0];\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1884)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:831)\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:37)\n\tat org.elasticsearch.common.Numbers.bytesToLong(Numbers.java:44)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2556)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2547)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.doGetRepositoryData(BlobStoreRepository.java:1881)\n\t... 6 more\n"
},
"status" : 500
}
hi guys, any ways to solve this?
system
(system)
Closed
October 20, 2021, 4:44pm
16
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.