ES 7.1 repository type [azure] does not exist

[jperezvt (Jesse Perez)]

I'm running ES 7.1 in Azure govcloud, its working fine and now wish to setting snapshots. Here is what I've done

Install plugin

PLUGIN_BIN=/usr/share/elasticsearch/bin/elasticsearch-plugin
echo "y" | $PLUGIN_BIN install repository-azure
and restarting ES. Now curl 0:9200/_cat/plugins?v showing es-cluster-vm0 repository-azure 7.1.1
Next, defined my storage setting via
export ES_KEYSTORE="/usr/share/elasticsearch/bin/elasticsearch-keystore"
export STORAGE_ACC_NAME="redacted"
export STORAGE_ACC_KEY="redacted"
echo {STORAGE_ACC_NAME} | {ES_KEYSTORE} add azure.client.default.account
echo {STORAGE_ACC_KEY} | {ES_KEYSTORE} add azure.client.default.key
echo "core.usgovcloudapi.net" | ${ES_KEYSTORE} add azure.client.default.endpoint_suffix

Next, created new container named es-snapshots with private access.

Next, to define my snapshot config ran
curl -XPUT 0:9200/_snapshot/azure_prod -H'Content-Type: application/json' -d '{ "type": "azure", "settings": { "container": "es-snapshots", "base_path": "backups", "chunk_size": "32m", "compress": true } }'

but getting error
{"error":{"root_cause":[{"type":"repository_exception","reason":"[azure_prod] repository type [azure] does not exist"}],"type":"repository_exception","reason":"[azure_prod] repository type [azure] does not exist"},"status":500}

Would really appreciate some clues/hints. Thanks, Jesse

[Armin_Braun]

Hi @jperezvt

did you install the Azure repository plugin on all your cluster nodes? The exception suggestions that its missing from some or all of your nodes. Since you say it shows up properly in the plugins list, it seems likely it's missing from a subset of your nodes.

[jperezvt (Jesse Perez)]

Thanks Armin, I had only installed Azure repository plugin on one of the master nodes for testing, did not realized I needed it on all nodes, doing that now, thanks

[jperezvt (Jesse Perez)]

My cluster consist of 4 nodes so I went ahead and setup Azure repository plugin on the rest of the nodes and restarted ES on each. Now when I try curl -XPUT 0:9200/_snapshot/azure_prod -H'Content-Type: application/json' -d '{ "type": "azure" }' I'm getting

{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[azure_prod] path is not accessible on master node"}],"type":"repository_verification_exception","reason":"[azure_prod] path is not accessible on master node","caused_by":{"type":"i_o_exception","reason":"Can not write blob master.dat","caused_by":{"type":"storage_exception","reason":"","caused_by":{"type":"unknown_host_exception","reason":"vtstorcoreprod.blob.core.windows.net"}}}},"status":500}

On each node I did set endpoint like echo "core.usgovcloudapi.net" | ${ES_KEYSTORE} add azure.client.default.endpoint_suffix since this is Azure govcloud (not commercial) and in the error above its still trying to use core.windows.net
Any other suggestion? Thanks, Jesse

[Armin_Braun]

@jperezvt

On each node I did set endpoint like echo "core.usgovcloudapi.net" | ${ES_KEYSTORE} add azure.client.default.endpoint_suffix since this is Azure govcloud (not commercial) and in the error above its still trying to use core.windows.net

You'll have to put the azure.client.default.endpoint_suffix setting in the elasticsearch.yml instead of into the secure settings. It's just a standard setting. See docs here.

[jperezvt (Jesse Perez)]

Thank you. I was confused from the online doc.