we have a cluster with 33 nodes (8 of which are data nodes) that shows a strange behaviour on one of the instances (named esdn3_1-v6).
At 1.00 in the night, this instance seems to take almost all the write load of the cluster, and the write queues of the instance fill up to 200.
If we stop the node, the load re-balances.
You can find attached the node stats, hot threads and tasks when the system is in this state.
The configuration files of the various nodes are identical.
What may be generating this behaviour?
(I can only upload the log files on wetransfer)