I'm running ES 7.1 in Azure govcloud, its working fine and now wish to setting snapshots. Here is what I've done
Install plugin
PLUGIN_BIN=/usr/share/elasticsearch/bin/elasticsearch-plugin echo "y" | $PLUGIN_BIN install repository-azure
and restarting ES. Now curl 0:9200/_cat/plugins?v showing es-cluster-vm0 repository-azure 7.1.1
Next, defined my storage setting via
export ES_KEYSTORE="/usr/share/elasticsearch/bin/elasticsearch-keystore"
export STORAGE_ACC_NAME="redacted"
export STORAGE_ACC_KEY="redacted"
echo {STORAGE_ACC_NAME} | {ES_KEYSTORE} add azure.client.default.account
echo {STORAGE_ACC_KEY} | {ES_KEYSTORE} add azure.client.default.key
echo "core.usgovcloudapi.net" | ${ES_KEYSTORE} add azure.client.default.endpoint_suffix
Next, created new container named es-snapshots with private access.
Next, to define my snapshot config ran curl -XPUT 0:9200/_snapshot/azure_prod -H'Content-Type: application/json' -d '{ "type": "azure", "settings": { "container": "es-snapshots", "base_path": "backups", "chunk_size": "32m", "compress": true } }'
but getting error {"error":{"root_cause":[{"type":"repository_exception","reason":"[azure_prod] repository type [azure] does not exist"}],"type":"repository_exception","reason":"[azure_prod] repository type [azure] does not exist"},"status":500}
Would really appreciate some clues/hints. Thanks, Jesse
did you install the Azure repository plugin on all your cluster nodes? The exception suggestions that its missing from some or all of your nodes. Since you say it shows up properly in the plugins list, it seems likely it's missing from a subset of your nodes.
Thanks Armin, I had only installed Azure repository plugin on one of the master nodes for testing, did not realized I needed it on all nodes, doing that now, thanks
My cluster consist of 4 nodes so I went ahead and setup Azure repository plugin on the rest of the nodes and restarted ES on each. Now when I try curl -XPUT 0:9200/_snapshot/azure_prod -H'Content-Type: application/json' -d '{ "type": "azure" }' I'm getting
{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[azure_prod] path is not accessible on master node"}],"type":"repository_verification_exception","reason":"[azure_prod] path is not accessible on master node","caused_by":{"type":"i_o_exception","reason":"Can not write blob master.dat","caused_by":{"type":"storage_exception","reason":"","caused_by":{"type":"unknown_host_exception","reason":"vtstorcoreprod.blob.core.windows.net"}}}},"status":500}
On each node I did set endpoint like echo "core.usgovcloudapi.net" | ${ES_KEYSTORE} add azure.client.default.endpoint_suffix since this is Azure govcloud (not commercial) and in the error above its still trying to use core.windows.net
Any other suggestion? Thanks, Jesse
You'll have to put the azure.client.default.endpoint_suffix setting in the elasticsearch.yml instead of into the secure settings. It's just a standard setting. See docs here.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.