Limiting traffic speed

Elastic Stack Elasticsearch
1

When transferring shards, the network speed between 2 nodes can increase up to 16 G/s. I want to limit network usage to 7G/s for the entire cluster.
Can this be done by means of ElasticSearch?

The only network regulator I could find is indices.recovery.max_bytes_per_sec, but it doesn't help

2

The default maximum speed is made up of indices.recovery.max_bytes_per_sec which defaults at 40mb/s per index, and this is multiplied by cluster.routing.allocation.node_concurrent_recoveries which default to 2 (incoming or outgoing), so once cluster is green and rebalance is needed/started, the default speed for shard rebalancing should be around 80Mb/s (2 outgoing or incoming * 40mb/s)

3

I don't think that's right,
indices.recovery.max_bytes_per_sec applies in aggregate across all recoveries. Adding concurrent recoveries will often lengthen the overall recovery time.

1 Like
4

Thank you so 40mb is limit per node shared by all recovery per node, concurrency is detrimental especially on spinning disks so best to increase speed than concurrency

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.