I am after a more in depth explanation of the implications of adjusting the max_snapshot_bytes_per_sec for an S3 repository. Reading the documentation, it is simply described as
max_snapshot_bytes_per_sec (Optional, byte value) Maximum snapshot creation rate per node. Defaults to 40mb per second.
My main questions are:
- Does adjusting this setting have a material impact on the cluster? ie: If I set this to way too high, is the Snapshotting process going to consume more resources and start to block index and search operations? I have found forum posts on here and stack overflow recommending increasing it (some increasing it to 10G).
- Does changing this setting impact running snapshots, or only new ones?
I have a cluster with 4 data nodes (to expand to probably 6) and around 20 TB of data (to expand to ~40) and want to ensure I am getting sufficient snapshot throughput to actually have a functional backup. I forgot to adjust my snapshot frequency as the data volumes have increased, and as a result I have spent 2 days just trying to get snapshots completing again and would like to know if adjusting this setting is a recommended approach to improving throughput.