Need clarity on recovery throttling parameters impacting snapshot restores

Elastic Stack Elasticsearch
1

Hi,
Current version used - elasticsearch-oss 7.8.0
Upgrading to: elasticsearch-oss 7.10.2

Elasticsearch 7.9 breaking change mentions about Snapshot restore throttling. As per the change -

  • Restoring from a snapshot now properly takes recovery throttling into account (i.e. the indices.recovery.max_bytes_per_sec setting).
  • max_restore_bytes_per_sec setting changed from 40mb default to unlimited.

Can someone help to understand the following:

  1. Is the parameter max_restore_bytes_per_sec only applicable to remote repositories like S3 or GCS as i can find the mention of this parameter only in snapshot/restore plugins. I use local glusterfs volume attached to store the snapshots. Will this setting still be applicable?

  2. How do both these parameters indices.recovery.max_bytes_per_sec and max_restore_bytes_per_sec work together in throttling the snapshot restore operation in versions>=7.9?

  3. What was the behavior before 7.9? Was there no throttling at all and restore could run at any rate as per the given resources? If yes:

  • The breaking change states that - This means that no behavioral change will be observed by clusters where the recovery and restore settings had not been adapted from the defaults.

  • If throttling was not considered at all previously (i.e. versions <7.9), restore would happen at a higher speed as per available resources.
    Now due to indices.recovery.max_bytes_per_sec=40mb limit getting considered in later versions, restores would run much slower (40mbps). How is it not a behavioral change?

  • How is the change in default value of max_restore_bytes_per_sec from 40mb to unlimited not causing a behavioral change?

Any pointers would be appreciated.
Thanks!

2

No, it applies to all restores.

The minimum of the two applies.

max_restore_bytes_per_sec applied, with a default of 40mb.

Because the effective limit is 40MBps either way.

Because indices.recovery.max_bytes_per_sec now applies.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.