# Unable to create snapshot repo in azure

**URL:** <https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804>\
**Category:** Elasticsearch\
**Tags:** snapshot-and-restore\
**Created:** [October 17, 2022, 2:49pm UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804 "2022-10-17T14:49:14Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![jaykb77](https://avatars.discourse-cdn.com/v4/letter/j/ac8455/32.png) [@jaykb77](https://discuss.elastic.co/u/jaykb77)\
**Post date:** [October 17, 2022, 2:49pm UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804/1 "2022-10-17T14:49:14Z")

</div>

Im trying to register an azure blob as snapshot repository. I have already tested this with elasticsearch 7.17 on test VMs outside my firm but it is failing when Im trying the same steps inside the firm.

My first impression was that a firewall connectivity issue with azure from our onpremise servers. But when I tested connection (\<storage\_account.\>[blob.core.windows.net:443](http://blob.core.windows.net:443)), it looked fine.

Im not sure if any other ports are required for this to work and the documentation does not mention anything related to the connection requirements. Can some one please help?

Note:

- I have added azure.client.default.account and azure.client.default.key on all nodes and the way it was added is already verified as working on test VMs outside my firm.
- I have also restarted the service on all nodes after the above keystore changes

Errors seen in API response and logs

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "settings_exception",
        "reason": "Unable to find client with name [default]"
      }
    ],
    "type": "repository_verification_exception",
    "reason": "[azure_repo] path is not accessible on master node",
    "caused_by": {
      "type": "settings_exception",
      "reason": "Unable to find client with name [default]"
    }
  },
  "status": 500
}

[2022-10-17T10:50:12,782][WARN][r.suppressed] [test-cluster-es01] path: /_snapshot/azure_repo, params: {repository=azure_repo}
org.elasticsearch.repositories.RepositoryVerificationException: [azure_repo] path is not accessible on master node
Caused by: org.elasticsearch.common.settings.SettingsException: Unable to find client with name [default]
        at org.elasticsearch.repositories.azure.AzureStorageService.getClientSettings(AzureStorageService.java:101) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureStorageService.client(AzureStorageService.java:91) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureBlobStore.getAzureBlobServiceClientClient(AzureBlobStore.java:624) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureBlobStore.asyncClient(AzureBlobStore.java:620) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureBlobStore.lambda$executeSingleUpload$19(AzureBlobStore.java:454) ~[?:?]
        at org.elasticsearch.repositories.azure.SocketAccess.lambda$doPrivilegedVoidException$0(SocketAccess.java:46) ~[?:?]
        at java.security.AccessController.doPrivileged(AccessController.java:569) ~[?:?]
        at org.elasticsearch.repositories.azure.SocketAccess.doPrivilegedVoidException(SocketAccess.java:45) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureBlobStore.executeSingleUpload(AzureBlobStore.java:453) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureBlobStore.writeBlob(AzureBlobStore.java:378) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureBlobContainer.writeBlob(AzureBlobContainer.java:103) ~[?:?]
        at org.elasticsearch.repositories.azure.AzureBlobContainer.writeBlobAtomic(AzureBlobContainer.java:98) ~[?:?]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:1677) ~[elasticsearch-7.17.0.jar:7.17.0]
        at org.elasticsearch.repositories.RepositoriesService$4.doRun(RepositoriesService.java:392) ~[elasticsearch-7.17.0.jar:7.17.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:777) [elasticsearch-7.17.0.jar:7.17.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.17.0.jar:7.17.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 17, 2022, 6:38pm UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804/2 "2022-10-17T18:38:07Z")

</div>

You should doubleck check if the config is really set in all nodes, your error pretty clear indicates that it is not

> [2022-10-17T10:50:12,782][WARN][r.suppressed] [test-cluster-es01] path: /\_snapshot/azure\_repo, params: {repository=azure\_repo}  
> org.elasticsearch.repositories.RepositoryVerificationException: [azure\_repo] path is not accessible on master node  
> Caused by: org.elasticsearch.common.settings.SettingsException: **Unable to find client with name [default]**

Check the keystore in every node to confirm that the configurations are really there.

---

<div class="post-metadata">

**Author:** ![jaykb77](https://avatars.discourse-cdn.com/v4/letter/j/ac8455/32.png) [@jaykb77](https://discuss.elastic.co/u/jaykb77)\
**Post date:** [October 18, 2022, 6:44am UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804/3 "2022-10-18T06:44:17Z")

</div>

Thanks for your reply. However, I have already verified this and double confirmed again.

I ran below steps on all nodes including 'elasticsearch-keystore show' to verify the configuration is set on all nodes.

```auto
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# ./elasticsearch-plugin list
ingest-attachment
repository-azure
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# STORAGE_ACCOUNT="elasticbkptest"
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# ACCESS_KEY="XXXXXXXXXXXXXXXX/4++ASt15ktPA=="
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin#   
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# echo $STORAGE_ACCOUNT | ./elasticsearch-keystore add -f --stdin azure.client.default.account
SS_KEY | ./elasticsearch-keystore add -f --stdin azure.client.default.key
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# echo $ACCESS_KEY | ./elasticsearch-keystore add -f --stdin azure.client.default.key
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# ./elasticsearch-keystore show azure.client.default.account
elasticbkptest
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# ./elasticsearch-keystore show azure.client.default.key    
XXXXXXXXXXXXXXXX/4++ASt15ktPA==
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# /etc/init.d/elasticsearch restart
[ok] Restarting elasticsearch (via systemctl): elasticsearch.service.
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 

```

Then ran the repo registration call and still I have this error

```auto
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# curl -k -u elastic -X PUT "https://localhost:9200/_snapshot/azure_backup" -H 'Content-Type: application/json' -d'
> {
> "type": "azure",
> "settings": {
> "container": "testbackup",
> "chunk_size": "32MB",
> "compress": true
> }
> }'
Enter host password for user 'elastic':
{"error":{"root_cause":[{"type":"settings_exception","reason":"Unable to find client with name [default]"}],"type":"repository_verification_exception","reason":"[azure_backup] path is not accessible on master node","caused_by":{"type":"settings_exception","reason":"Unable to find client with name [default]"}},"status":500}root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# 
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# curl -k -u elastic -X POST ""https://localhost:9200/_snapshot/azure_backup/_verify?pretty"
> ^C
root@test-search-es01.qaextranet:/a/elasticsearch/elasticsearch-current/bin# curl -k -u elastic -X POST "https://localhost:9200/_snapshot/azure_backup/_verify?pretty"
Enter host password for user 'elastic':
{
  "error" : {
    "root_cause" : [
      {
        "type" : "settings_exception",
        "reason" : "Unable to find client with name [default]"
      }
    ],
    "type" : "repository_verification_exception",
    "reason" : "[azure_backup] path is not accessible on master node",
    "caused_by" : {
      "type" : "settings_exception",
      "reason" : "Unable to find client with name [default]"
    }
  },
  "status" : 500
}

```

---

<div class="post-metadata">

**Author:** ![jaykb77](https://avatars.discourse-cdn.com/v4/letter/j/ac8455/32.png) [@jaykb77](https://discuss.elastic.co/u/jaykb77)\
**Post date:** [October 18, 2022, 1:14pm UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804/4 "2022-10-18T13:14:42Z")

</div>

Im afraid if its related to how elasticsearch is installed on the machines inside the firm. We used elasticsearch ansible role for the deployment and the config folder used is non-standard(es\_conf\_dir as per ansible below). So the keystore file that created for the xpack during deployment is located in a non default path.

Does this affect adding azure repo settings? It doesnt make sense since I am anyway able to list and show the azure repo settings in keystore after adding it though.

Even if we have a custom config location, we should be able to add keystore settings to file in that location - but I can not see such options with the keystore command.

> <https://github.com/elastic/ansible-elasticsearch/blob/main/tasks/xpack/security/elasticsearch-security.yml>

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 18, 2022, 1:26pm UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804/5 "2022-10-18T13:26:59Z")

</div>

I'm not sure, but I think that the `elasticsearch.keystore` file needs to be in the `ES_PATH_CONF`.

Is the `elasticsearc.keystore` in the same path for `ES_PATH_CONF` ? The same place where `elasticsearch.yml` is located.

---

<div class="post-metadata">

**Author:** ![jaykb77](https://avatars.discourse-cdn.com/v4/letter/j/ac8455/32.png) [@jaykb77](https://discuss.elastic.co/u/jaykb77)\
**Post date:** [October 18, 2022, 1:33pm UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804/6 "2022-10-18T13:33:58Z")

</div>

Okay, I fixed this now. I had to export ES\_PATH\_CONF (since we were using a custom config location) before running keystore commands so that the keystore file matches with what elasticsearch service uses.  
Thank you for your time and support 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 15, 2022, 1:34pm UTC](https://discuss.elastic.co/t/unable-to-create-snapshot-repo-in-azure/316804/7 "2022-11-15T13:34:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
