# Elasticsearch snapshots throttle problems

**URL:** <https://discuss.elastic.co/t/elasticsearch-snapshots-throttle-problems/21052>\
**Category:** Elasticsearch\
**Created:** [December 3, 2014, 9:25am UTC](https://discuss.elastic.co/t/elasticsearch-snapshots-throttle-problems/21052 "2014-12-03T09:25:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Johan\_Ohr](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@Johan\_Ohr](https://discuss.elastic.co/u/Johan_Ohr)\
**Post date:** [December 3, 2014, 9:25am UTC](https://discuss.elastic.co/t/elasticsearch-snapshots-throttle-problems/21052/1 "2014-12-03T09:25:38Z")

</div>

Hi,

I have 12 elasticsearch nodes, with 10gb eth

Ive been having alot of problem with the performance of snapshots, its  
throttles to 20 mb/s even tho i set max\_snapshot\_bytes\_per\_sec to something  
else, ive tried to set it in bytes, in megabytes (500m, 500mb)

Ive tried to move 100gb file from elastic-node to my nfs-server, its about  
1gb/s  
Ive tried to move 100gb file from elastic-node down to my share, its about  
1gb/s  
Ive tried to just cp -rp my index from elastic-node to my share, its about  
1gb/s

Am i missing something here? How is the max\_snapshot\_bytes\_per\_sec suppose  
to look like?  
Are there any other settings (like recovery streams etc) that affects this?

My backup dir:  
curl -XPUT '[http://localhost:9200/\_snapshot/my\_backup](http://localhost:9200/_snapshot/my_backup)' -d '{  
"type": "fs",  
"settings": {  
"location": "/misc/backup\_elastic/snapshot",  
"compress": true,  
"verify": true,  
"max\_snapshot\_bytes\_per\_sec" : "1048576000",  
"max\_restore\_bytes\_per\_sec" : "1048576000"  
}  
}'

And snapshot:  
curl -XPUT "localhost:9200/\_snapshot/my\_backup/snapshot\_test" -d '{  
"indices": "index-2014-01-03",  
"ignore\_unavailable": "true",  
"include\_global\_state": "true",  
"partial": "true"  
}'

Regards, Johan

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/fcb8e013-6174-44f7-9343-5090665c43f1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fcb8e013-6174-44f7-9343-5090665c43f1%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Johan\_Ohr](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@Johan\_Ohr](https://discuss.elastic.co/u/Johan_Ohr)\
**Post date:** [December 4, 2014, 8:21am UTC](https://discuss.elastic.co/t/elasticsearch-snapshots-throttle-problems/21052/2 "2014-12-04T08:21:45Z")

</div>

I noticed these warnings on some of my nodes while executing the snapshot,  
maybe it has to do something with why its so slow.

[2014-12-03 15:57:35,699][WARN][snapshots] [xx06]  
[[xxx-2014-11-20][7]] [my\_backup:snapshot\_test] failed to create snapshot

org.elasticsearch.index.snapshots.IndexShardSnapshotFailedException:  
[xxx-2014-11-20][7] Aborted

at  
org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext$AbortableInputStream.checkAborted(BlobStoreIndexShardRepository.java:632)

at  
org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext$AbortableInputStream.read(BlobStoreIndexShardRepository.java:625)

at java.io.FilterInputStream.read(FilterInputStream.java:107)

at  
org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext.snapshotFile(BlobStoreIndexShardRepository.java:557)

at  
org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext.snapshot(BlobStoreIndexShardRepository.java:501)

at  
org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:139)

at  
org.elasticsearch.index.snapshots.IndexShardSnapshotAndRestoreService.snapshot(IndexShardSnapshotAndRestoreService.java:86)

at  
org.elasticsearch.snapshots.SnapshotsService$5.run(SnapshotsService.java:818)

at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745

Den onsdagen den 3:e december 2014 kl. 10:25:38 UTC+1 skrev Johan Öhr:

> Hi,
> 
> I have 12 elasticsearch nodes, with 10gb eth
> 
> Ive been having alot of problem with the performance of snapshots, its  
> throttles to 20 mb/s even tho i set max\_snapshot\_bytes\_per\_sec to something  
> else, ive tried to set it in bytes, in megabytes (500m, 500mb)
> 
> Ive tried to move 100gb file from elastic-node to my nfs-server, its about  
> 1gb/s  
> Ive tried to move 100gb file from elastic-node down to my share, its about  
> 1gb/s  
> Ive tried to just cp -rp my index from elastic-node to my share, its about  
> 1gb/s
> 
> Am i missing something here? How is the max\_snapshot\_bytes\_per\_sec suppose  
> to look like?  
> Are there any other settings (like recovery streams etc) that affects  
> this?
> 
> My backup dir:  
> curl -XPUT '[http://localhost:9200/\_snapshot/my\_backup](http://localhost:9200/_snapshot/my_backup)' -d '{  
> "type": "fs",  
> "settings": {  
> "location": "/misc/backup\_elastic/snapshot",  
> "compress": true,  
> "verify": true,  
> "max\_snapshot\_bytes\_per\_sec" : "1048576000",  
> "max\_restore\_bytes\_per\_sec" : "1048576000"  
> }  
> }'
> 
> And snapshot:  
> curl -XPUT "localhost:9200/\_snapshot/my\_backup/snapshot\_test" -d '{  
> "indices": "index-2014-01-03",  
> "ignore\_unavailable": "true",  
> "include\_global\_state": "true",  
> "partial": "true"  
> }'
> 
> Regards, Johan

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/13857ee7-5e36-45a5-a2b5-3ffa66e8fa21%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/13857ee7-5e36-45a5-a2b5-3ffa66e8fa21%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![mhughes](https://avatars.discourse-cdn.com/v4/letter/m/85e7bf/32.png) [@mhughes](https://discuss.elastic.co/u/mhughes)\
**Post date:** [December 9, 2014, 1:34am UTC](https://discuss.elastic.co/t/elasticsearch-snapshots-throttle-problems/21052/3 "2014-12-09T01:34:53Z")

</div>

Those settings look correct to me. You can set using kb,mb,gb, etc.

On Wednesday, December 3, 2014 4:25:38 AM UTC-5, Johan Öhr wrote:

> Hi,
> 
> I have 12 elasticsearch nodes, with 10gb eth
> 
> Ive been having alot of problem with the performance of snapshots, its  
> throttles to 20 mb/s even tho i set max\_snapshot\_bytes\_per\_sec to something  
> else, ive tried to set it in bytes, in megabytes (500m, 500mb)
> 
> Ive tried to move 100gb file from elastic-node to my nfs-server, its about  
> 1gb/s  
> Ive tried to move 100gb file from elastic-node down to my share, its about  
> 1gb/s  
> Ive tried to just cp -rp my index from elastic-node to my share, its about  
> 1gb/s
> 
> Am i missing something here? How is the max\_snapshot\_bytes\_per\_sec suppose  
> to look like?  
> Are there any other settings (like recovery streams etc) that affects  
> this?
> 
> My backup dir:  
> curl -XPUT '[http://localhost:9200/\_snapshot/my\_backup](http://localhost:9200/_snapshot/my_backup)' -d '{  
> "type": "fs",  
> "settings": {  
> "location": "/misc/backup\_elastic/snapshot",  
> "compress": true,  
> "verify": true,  
> "max\_snapshot\_bytes\_per\_sec" : "1048576000",  
> "max\_restore\_bytes\_per\_sec" : "1048576000"  
> }  
> }'
> 
> And snapshot:  
> curl -XPUT "localhost:9200/\_snapshot/my\_backup/snapshot\_test" -d '{  
> "indices": "index-2014-01-03",  
> "ignore\_unavailable": "true",  
> "include\_global\_state": "true",  
> "partial": "true"  
> }'
> 
> Regards, Johan

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5b753ce0-42d1-4b47-aab4-4d021bcaf555%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/5b753ce0-42d1-4b47-aab4-4d021bcaf555%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:44am UTC](https://discuss.elastic.co/t/elasticsearch-snapshots-throttle-problems/21052/4 "2017-07-06T00:44:47Z")

</div>