We are currently having a very large cluster with about 100 TB data in it. We want our snapshot in Amazon S3 Bucket to complete as fast as possible.
With reference from this
I tried increasing the maximum number of threads in the snapshot thread pool. The number of active threads increased during the Snapshotting process without any increase in the Snapshotting speed.
Currently I am getting 200 MBps Speed of Snapshot/Restore the data onto S3 Bucket
I have checked out that my S3 Bucket has far more limits (Around 2 GBps) than this.
Also I when I am doing a restore of the a snapshot from a Amazon S3, I am able to see threads active from the Generic Thread pool, instead of being active from the Snapshot thread pool.
What could be wrong here?
Thanks in advance!