I'm working on setting up an elastic search environment within a kubernetes cluster (on MS Azure AKS), and am reading through the Elastic Cloud on Kubernetes [1.5] | Elastic documentation. It explains how to setup clusters and configure them.
The cluster will be used for gather logs of our various microservices running inside the kubernetes cluster (they are .NET services and will be sending their logs using the Serilog elastic search sink), and, more importantly, indexing various models of our application specific data for fast searching. Lastly, I also want to monitor the performance of all clusters we end up deploying.
From the Monitoring overview | Elasticsearch Guide [7.12] | Elastic docs, I gathered that monitoring of an elastic search cluster, should be done in a separate monitoring cluster. This got me thinking, what else, besides monitoring, constitutes the need for a separate cluster. A cluster can have multiple indices, so can everything else (logs & search indices) be put in one cluster, or is there some guidelines as to what should be put in a separate cluster?
I could not find this information anywhere. There's a lot to be found about node planning and shard sizing, but not about clusters in general.