# How to plan/define needed clusters

**URL:** <https://discuss.elastic.co/t/how-to-plan-define-needed-clusters/273514>\
**Category:** Elasticsearch\
**Created:** [May 20, 2021, 11:10am UTC](https://discuss.elastic.co/t/how-to-plan-define-needed-clusters/273514 "2021-05-20T11:10:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![agKaspar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/agkaspar/32/101578_2.png) [@agKaspar](https://discuss.elastic.co/u/agKaspar)\
**Post date:** [May 20, 2021, 11:10am UTC](https://discuss.elastic.co/t/how-to-plan-define-needed-clusters/273514/1 "2021-05-20T11:10:09Z")

</div>

Hello,

I'm working on setting up an elastic search environment within a kubernetes cluster (on MS Azure AKS), and am reading through the [Elastic Cloud on Kubernetes [1.5] | Elastic](https://www.elastic.co/guide/en/cloud-on-k8s/current/index.html) documentation. It explains how to setup clusters and configure them.

The cluster will be used for gather logs of our various microservices running inside the kubernetes cluster (they are .NET services and will be sending their logs using the Serilog elastic search sink), and, more importantly, indexing various models of our application specific data for fast searching. Lastly, I also want to monitor the performance of all clusters we end up deploying.

From the [Monitoring overview | Elasticsearch Guide [7.12] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/monitoring-overview.html) docs, I gathered that monitoring of an elastic search cluster, should be done in a separate monitoring cluster. This got me thinking, what else, besides monitoring, constitutes the need for a separate cluster. A cluster can have multiple indices, so can everything else (logs & search indices) be put in one cluster, or is there some guidelines as to what should be put in a separate cluster?

I could not find this information anywhere. There's a lot to be found about node planning and shard sizing, but not about clusters in general.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 25, 2021, 12:25am UTC](https://discuss.elastic.co/t/how-to-plan-define-needed-clusters/273514/2 "2021-05-25T00:25:16Z")

</div>

> [@agKaspar](#):
>
> I gathered that monitoring of an Elasticsearch cluster, should be done in a separate monitoring cluster

It's a best practise, but it's not mandatory.

> [@agKaspar](#):
>
> This got me thinking, what else, besides monitoring, constitutes the need for a separate cluster. A cluster can have multiple indices, so can everything else (logs & search indices) be put in one cluster, or is there some guidelines as to what should be put in a separate cluster?

Usually you wouldn't put logs (text) and search data in the same cluster.

You might also separate prod and non-prod clusters.  
You can also consider moving high velocity data into it's own cluster with different specs to others.  
And you might also want to keep data geolocated to it's source, to save data transfer costs

The last three are optional.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 22, 2021, 12:26am UTC](https://discuss.elastic.co/t/how-to-plan-define-needed-clusters/273514/3 "2021-06-22T00:26:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.