How to correctly determine the weight of a node

Elastic Stack Elasticsearch
1

Hey Team,

I'm trying to calculate the weights of each node based on the below two settings taken from shard balancing heuristics:

cluster.routing.allocation.balance.shard
cluster.routing.allocation.balance.index

Let's say we have the below scenario:

Node 1 has 2 indexes: index A and index B
Node 1 also has 3 primary shards and 3 replica shards (6 shards in total).
cluster.routing.allocation.balance.shard = 0.45f
cluster.routing.allocation.balance.index = 0.55f

Then as per the documentation:

  1. cluster.routing.allocation.balance.shard defines the weight factor for the total number of shards allocated on a node:

    Total number of shards * 0.45 = 6 * 0.45 = 2.7

  2. cluster.routing.allocation.balance.index defines the weight factor for the number of shards per index allocated on a specific node:

    Number of shards per index = 6 (total number of shards) / 2 (number of indexes on the node) = 3.0

  3. Weight of Node 1 is the sum of weights calculated from 1. and 2.:

    Node 1 weight = 2.7 + 3 = 5.7

Is the above a correct way of calculating a node's weight?

Cheers,

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

3

Hey there,

I was wondering the same thing, and went on a little trip into the rabbit hole.

There you have the code for the WeightFunction class that compute the weight of each nodes.

And as you can see the weight function does the computation on 4 characteristics of a node:

  • ShardsLoad = Number of shard on the node - Avg number of shards per node in the cluster
  • IndexLoad = Number of shard for a given index - Avg number of shards for a given index in the cluster
  • WriteLoad = Write load on a node - Avg write load
  • DiskLoad = Actual disk usage - Avg disk usage in the cluster

All of those are then multiplied by their respective settings and divided by the sum of all settings.

SumSettings = cluster.routing.allocation.balance.shard + cluster.routing.allocation.balance.index + cluster.routing.allocation.balance.disk_usage + cluster.routing.allocation.balance.write_load

So in the end you have:

Weight = ShardsLoad * cluster.routing.allocation.balance.shard / SumSettings
+ IndexLoad * cluster.routing.allocation.balance.index / SumSettings
+ WriteLoad * cluster.routing.allocation.balance.write_load / SumSettings
+ DiskLoad * cluster.routing.allocation.balance.disk_usage / SumSettings

1 Like