Hi!
Our organization is implementing a new monitor system and we want to use the Elastic Stack taking advantage of our previous experience using it as a log system for troubleshooting.
Beside collect data, one of the most important parts of the system will be the alerting modules, watchers. Our, main concern regarding the watchers, it's own it scale for a large number of environments (group of machines), we want to be able to monitor 20 to 30 metrics by each environment.
I had read this article regarding the distributed watch execution
https://www.elastic.co/blog/distributed-watch-execution-elasticsearch-6.0
My question is: "We will able to create 150 000 watchers running in 1minute intervals? ( 30 metrics * 5000 envs )" and another 150K in the next year? There is a recommended limit? And the performance?
Any advice? Places to look?
Thanks much in advance!