We are running ES cluster as kuberntes pods in google cloud environment. We are planning to
trigger the snapshots and need to store the data GCS bucket.
As part of snapshot, we are creating the rep0, but the repo creation is always failing with below error. But it's internally creating the ES repo successfully, seems to be deletion of stale data from bucket is failing.
Suprisingly, with this repo name, we can able to trigger the snapshot and deletion API's successfully in that cluster.
we have given complete storage admin access to the VM's where ES is running. can you please help us, why repo creation is failing with below error?
curl -XPUT -H "Content-Type: application/json" -d '{
"type": "gcs",
"settings":
{
"bucket": "test-abc",
"base_path": "es_backup1",
}
}' localhost:9200/_snapshot/backup2
{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[backup] cannot delete test data at [es_backup]"}],"type":"repository_verification_exception","reason":"[backup] cannot delete test data at [es_backup]","caused_by":{"type":"i_o_exception","reason":"404 Not Found\n\n\n\n \n \n Error 404 (Not Found)!!1\n \n {margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px} > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}\n \n \n
404. That’s an error.\n
That’s all we know.\n"}},"status":500}