# Snapshot create \[Index Shard Snapshot Failed \] :: Acess Denied

**URL:** https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969
**Category:** Elasticsearch
**Created:** [January 25, 2018, 5:43am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969 "2018-01-25T05:43:53Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![hpandey](https://avatars.discourse-cdn.com/v4/letter/h/bbe5ce/32.png) [@hpandey](https://discuss.elastic.co/u/hpandey)
#### Post date: [January 25, 2018, 5:43am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/1 "2018-01-25T05:43:53Z")

</div>

```auto
Hi All 

Having issue while taking cluster snapshot . 
Env :  
suse 12 
5 node cluster 3M + 2D
elastic version : 5.5 
Backup type : fs 

Error :  
 "reason": "IndexShardSnapshotFailedException[Failed to snapshot]; nested: ElasticsearchException[failed to create blob container]; nested: AccessDeniedException[/backup/elasticsearch-backup/lb/indices/huUf9iZ9StCglJsOOsYs1A/0]; ",
          "node_id": "BYIubOVHR9G23ehNfEfZ7Q",
          "status": "INTERNAL_SERVER_ERROR"

Can somebody highlight or provide solution , why Access Denied exception is comming  
Thanks

```

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 25, 2018, 6:59am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/2 "2018-01-25T06:59:25Z")

</div>

Please format your code using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

Please edit your post.

You probably have a stacktrace in logs. Could you share it ?

---

<div class="post-metadata">

### Author: ![hpandey](https://avatars.discourse-cdn.com/v4/letter/h/bbe5ce/32.png) [@hpandey](https://discuss.elastic.co/u/hpandey)
#### Post date: [January 25, 2018, 7:51am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/3 "2018-01-25T07:51:17Z")

</div>

```auto
stacktrace  

[2018-01-25T02:38:52,066][WARN][o.e.s.SnapshotShardsService] [pc4cmelk02] [[.watcher-history-3-2017.12.26][0]] [lb_backup:lab_base1/pBNMVlaUSwGohP3nBsi0_g] failed to create snapshot
org.elasticsearch.index.snapshots.IndexShardSnapshotFailedException: Failed to snapshot
        at org.elasticsearch.snapshots.SnapshotShardsService.snapshot(SnapshotShardsService.java:397) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.snapshots.SnapshotShardsService.access$200(SnapshotShardsService.java:88) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.snapshots.SnapshotShardsService$1.doRun(SnapshotShardsService.java:335) [elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.5.0.jar:5.5.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: org.elasticsearch.ElasticsearchException: failed to create blob container
        at org.elasticsearch.common.blobstore.fs.FsBlobStore.blobContainer(FsBlobStore.java:70) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository$Context.<init>(BlobStoreRepository.java:1078) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository$Context.<init>(BlobStoreRepository.java:1071) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository$SnapshotContext.<init>(BlobStoreRepository.java:1284) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository.snapshotShard(BlobStoreRepository.java:968) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.snapshots.SnapshotShardsService.snapshot(SnapshotShardsService.java:382) ~[elasticsearch-5.5.0.jar:5.5.0]
        ... 7 more
Caused by: java.nio.file.AccessDeniedException: /backup/elasticsearch-backup/lb/indices/u-zKaqTYSqWeJNe1vmBQDQ/0
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_101]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_101]
        at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_101]
        at org.elasticsearch.common.blobstore.fs.FsBlobStore.buildAndCreate(FsBlobStore.java:86) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.common.blobstore.fs.FsBlobStore.blobContainer(FsBlobStore.java:68) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository$Context.<init>(BlobStoreRepository.java:1078) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository$Context.<init>(BlobStoreRepository.java:1071) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository$SnapshotContext.<init>(BlobStoreRepository.java:1284) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.repositories.blobstore.BlobStoreRepository.snapshotShard(BlobStoreRepository.java:968) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.snapshots.SnapshotShardsService.snapshot(SnapshotShardsService.java:382) ~[elasticsearch-5.5.0.jar:5.5.0]

```

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 25, 2018, 9:05am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/4 "2018-01-25T09:05:39Z")

</div>

```auto
java.nio.file.AccessDeniedException: /backup/elasticsearch-backup/lb/indices/u-zKaqTYSqWeJNe1vmBQDQ/0

```

One of your nodes doesn't have access to this dir apparently?

When this error is appearing? What are the commands that you sent?

---

<div class="post-metadata">

### Author: ![hpandey](https://avatars.discourse-cdn.com/v4/letter/h/bbe5ce/32.png) [@hpandey](https://discuss.elastic.co/u/hpandey)
#### Post date: [January 29, 2018, 5:54am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/5 "2018-01-29T05:54:00Z")

</div>

```auto
Hi  
All the node have elasticsearch user permission to the repo path "/backup/elasticsearch-backup/lb "
 while taking snaphot from "Cerebro v0.6.5" it creates the meta files ,but get the Access denied error while writing to the "index " folder 

Thanks 

```

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 29, 2018, 6:06am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/6 "2018-01-29T06:06:15Z")

</div>

Do you intentionally use this strange formatting for your text?

Could you `ls -l` your backup dir?

How many nodes you have?

---

<div class="post-metadata">

### Author: ![hpandey](https://avatars.discourse-cdn.com/v4/letter/h/bbe5ce/32.png) [@hpandey](https://discuss.elastic.co/u/hpandey)
#### Post date: [January 29, 2018, 6:22am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/7 "2018-01-29T06:22:46Z")

</div>

Hi  
ls -l output

```auto
ls -l /backup/elasticsearch-backup/lb/

-rw-r--r-- 1 488 488 29 Jan 23 09:20 incompatible-snapshots
-rw-r--r-- 1 488 488 29 Jan 25 02:37 index-1
-rw-r--r-- 1 488 488 5589 Jan 25 02:38 index-2
-rw-r--r-- 1 488 488 8 Jan 25 02:38 index.latest
drwxr-xr-x 56 488 488 8192 Jan 25 02:38 indices
-rw-r--r-- 1 488 488 15548 Jan 25 02:38 meta-pBNMVlaUSwGohP3nBsi0_g.dat
-rw-r--r-- 1 488 488 19886 Jan 25 02:38 snap-pBNMVlaUSwGohP3nBsi0_g.dat

```

Thanks

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 29, 2018, 7:26am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/8 "2018-01-29T07:26:28Z")

</div>

> How many nodes you have?

---

<div class="post-metadata">

### Author: ![hpandey](https://avatars.discourse-cdn.com/v4/letter/h/bbe5ce/32.png) [@hpandey](https://discuss.elastic.co/u/hpandey)
#### Post date: [January 29, 2018, 7:38am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/9 "2018-01-29T07:38:33Z")

</div>

Hi

5 nodes = 2 Data Node and 3 Master Node and X-Pack on top it .

Thanks  
Himanshu

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 29, 2018, 9:53am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/10 "2018-01-29T09:53:50Z")

</div>

Is `/backup/elasticsearch-backup/lb/` mount on every node and accessible via the user who is running the elasticsearch node locally.

Here I guess that one of the 5 nodes can not write there.

---

<div class="post-metadata">

### Author: ![hpandey](https://avatars.discourse-cdn.com/v4/letter/h/bbe5ce/32.png) [@hpandey](https://discuss.elastic.co/u/hpandey)
#### Post date: [January 29, 2018, 10:31am UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/11 "2018-01-29T10:31:34Z")

</div>

Hi

All the node have same mount point and elasticsearch user have permision on it.

Thanks

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 29, 2018, 12:51pm UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/12 "2018-01-29T12:51:45Z")

</div>

Is `elasticsearch` user actually user `488` as I can see in the `ls -l`?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 26, 2018, 1:05pm UTC](https://discuss.elastic.co/t/snapshot-create-index-shard-snapshot-failed-acess-denied/116969/13 "2018-02-26T13:05:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.