I have issue in the heartbeat when reload.enabled change to true.
The condition as follow:
- I have around 120 monitor yml file
- I created around 40 new monitor yml file
- Before the reload.enabled: true, the pipeline clients indicate the correct number
- Then, I thought if I make reload.enabled: true, I can add new monitor yml automatically read by heartbeat
- just copy the yml file to monitors.d folder
- The heartbeat log said there are new configuration as expected
- For several minutes, the pipeline client goes to more than 2000 and all metric of heartbeat going crazy
- I removed the new yml but the pipeline still high and higher
- restart the heartbeat and the pipeline client back to normal
When I revert to false, there is no issue anymore regarding the increasing the pipeline clients.