# Does creating snapshot make cluster slow? **URL:** \ **Category:** Elasticsearch\ **Created:** [October 12, 2018, 4:27am UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149 "2018-10-12T04:27:38Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Jianzhou\_Z](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jianzhou_z/32/46751_2.png) [@Jianzhou\_Z](https://discuss.elastic.co/u/Jianzhou_Z)\ **Post date:** [October 12, 2018, 4:27am UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149/1 "2018-10-12T04:27:38Z")
I use ElasticSearch 5.6. When running snapshot, I run [http://localhost:9200/\_cluster/health](http://localhost:9200/_cluster/health) but did not get response for more than 10 sec. I can also see when snapshot runs, machines have a lot of costs at disk/network IO. Such a delay does not happen if I do not run snapshot. I check \_cluster/health with timeout to ensure that creating snapshot does not slow-down queries. Is it the correct way to check this? In practice will creating snapshots make queries slow down? ---
**Author:** ![loren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/loren/32/44942_2.png) [@loren](https://discuss.elastic.co/u/loren)\ **Post date:** [October 12, 2018, 10:38pm UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149/2 "2018-10-12T22:38:14Z")
I imagine creating snapshots _could_ slow down your queries. Snapshots are going to read lots of blocks off disk, and those blocks will go into the filesystem cache. Queries that might have been served from FS cache may need to go to disk instead, and then they will be competing with the snapshot process for access to disk IO. The effect would be magnified for larger documents, large result sets, and scan/scrolls. ---
**Author:** ![Jianzhou\_Z](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jianzhou_z/32/46751_2.png) [@Jianzhou\_Z](https://discuss.elastic.co/u/Jianzhou_Z)\ **Post date:** [October 12, 2018, 10:52pm UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149/3 "2018-10-12T22:52:11Z")
@loren, Do we have any good practice to follow about when to run snapshot? I run snapshot for a running elasticsearch cluster that provides service to users. Ideally we may not want to affect users' search experience. One option is doing this when traffic is low. But it is still possible to that traffic changes suddenly... ---
**Author:** ![loren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/loren/32/44942_2.png) [@loren](https://discuss.elastic.co/u/loren)\ **Post date:** [October 12, 2018, 11:15pm UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149/4 "2018-10-12T23:15:24Z")
I don't know of anything other than spreading the data onto more nodes, or increasing the RAM of each node. Both of those will improve the ratio of RAM to disk, which would lessen the impact of FS cache purges and disk contention. [Better compression](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-codec) could help in the same way. But first I'd want to benchmark some more to be sure that snapshot is indeed interfering with query performance. And then I'd try to make the queries less reliant on disk reads, perhaps by storing fewer fields or breaking the data up into multiple indices. Good luck! ---
**Author:** ![Jianzhou\_Z](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jianzhou_z/32/46751_2.png) [@Jianzhou\_Z](https://discuss.elastic.co/u/Jianzhou_Z)\ **Post date:** [October 14, 2018, 4:40pm UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149/5 "2018-10-14T16:40:34Z")
Will old GC be the problem of the slowdown? I made another post: [Does GC at snapshot affect performance? can users force GC?](https://discuss.elastic.co/t/does-gc-at-snapshot-affect-performance-can-users-force-gc/152374) ---
**Author:** ![loren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/loren/32/44942_2.png) [@loren](https://discuss.elastic.co/u/loren)\ **Post date:** [October 15, 2018, 6:48pm UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149/6 "2018-10-15T18:48:31Z")
I doubt it, unless your JVM is under so much memory pressure that it has to stop the world. Doesn't seem to be the case for you. I'd suggest running something like `iostat` before/during/after your snapshot. If you see IO Wait times spike during the snapshot, then any queries not served 100% from memory are going to take longer. If you regularly see IO Wait when snapshot _isn't_ running, then you already have an overburdened storage system and you have bigger problems than snapshot. Make sure you run `iostat` on the volumes that contain your shards. For example, I might run `iostat -mx nvme0n1 11` to monitor how a mounted NVMe drive on my EC2 i3 instance is performing. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 12, 2018, 6:48pm UTC](https://discuss.elastic.co/t/does-creating-snapshot-make-cluster-slow/152149/7 "2018-11-12T18:48:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.