Hello @Kerry
Thank you for your response.
With my configurations above, the number of documents written per minutes are as follow.
| event.dataset | docs per minutes |
|---|---|
| system.diskio | 6 |
| system.fsstat | 6 |
| system.load | 6 |
| system.uptime | 6 |
| system.cpu | 6 |
| system.memory | 6 |
| system.process.summary | 6 |
| system.network | 96 |
| total | 138 |
So, I can get required write performance as follow.
- 2.3 tps per 1 node
- 4,600 tps for 2,000 nodes.
This write load should be handled with our elasticsearch cluster. (I've done over 30K write performance test on our environment. The use cases was different though)
What I want to know is Metric UI sides latency.
Since I got UI latency problem with using Elastic APM (THIS ISSUE, THIS ISSUE), and found out the team just started architectural improvement to solve latency problem.
So we just want to make it sure that scalability of Metric UI, before going further.
As you mentioned as follow, there won't be scalability problem with UI side.
Serving 2,000 nodes in the UI shouldn't be a problem
We will trying to apply Metricbeat more machines.