# Most right architecture for my cluster
**URL:** https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488
**Category:** Elasticsearch
**Created:** [January 31, 2019, 7:45am UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488 "2019-01-31T07:45:34Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Amit\_Ripshtos](https://discuss.elastic.co/u/Amit_Ripshtos)
#### Post date: [January 31, 2019, 7:45am UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488/1 "2019-01-31T07:45:34Z")
Hey all,
Currently I have a 3 nodes cluster with the following spec:
Every server has 12 cores, 32gb of memory and 3 TB of NVME disks.
I have total of 6.5TB of data, spreaded on 30 daily indices and another four indices with 10-200 million of documents.
Every day I bulk between 200-350GB of data.
Every daily index have 3 primary shards with no replicas.
Im having heap issues due to the amount of indexing we are doing, and overall slowness, therefore I order another 2 servers, with the same spec.
Now, I have some options regarding the amount of master nodes:
1. 5 nodes that are both master and data nodes
2. 3 master+data nodes and 2 data nodes
My main goal is to have a faster cluster (bulks are taking 1-15 seconds per bulk) and to have a stable cluster (my cluster is getting freezed every week due to OOM)
Thanks!
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 31, 2019, 8:21am UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488/2 "2019-01-31T08:21:13Z")
I think that you probably need a bit more shards for the current index if you want to speed up indexing.
Then the next day you might want to use shrink API to reduce that number.
May be you should add more nodes and think of a hot-warm design?
I'm dropping here some documentation which might be helpful.
[https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing)
> **[How many shards should I have in my Elasticsearch cluster?
| Elastic](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)**
>
> Elasticsearch is a very versatile platform, that supports a variety of use cases, and provides great flexibility around data organisation and replication strategies. This flexibility can however somet...
> **[NetSecureDay: Managing your Black Friday Logs](https://speakerdeck.com/elastic/netsecureday-managing-your-black-friday-logs)**
>
> Surveiller une application complexe n’est pas une tâche aisée, mais avec les bons outils, ce n’est pas si sorcier. Néanmoins, des périodes fortes telles que les opérations de type « Black Friday » (Vendredi noir) ou période de Noël peuvent pousser...
And [https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right](https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right)
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [January 31, 2019, 8:23am UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488/3 "2019-01-31T08:23:19Z")
[This webinar](https://www.elastic.co/webinars/optimizing-storage-efficiency-in-elasticsearch) might also be useful as you are suffering from heap pressure. A good way to consistently get reasonably sized shards is to use the [rollover API](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/indices-rollover-index.html). This allows to to create indices when needed based on age and/or size instead of doing this rigidly on a daily basis, and is supported by the new [Index Lifecycle Management feature](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/index-lifecycle-management.html) available in version 6.6. This will allow you to use the same number of primary shards as you have nodes, but cut indices more or less frequently than now depending on indexing volumes.
---
### Author:  [@juka](https://discuss.elastic.co/u/juka)
#### Post date: [January 31, 2019, 10:00am UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488/4 "2019-01-31T10:00:11Z")
Go for Option Two
3 master + 2 data nodes
You need backup so go for 1 replica. If a node fails your cluster will still function.
As mentioned above use the rollover API and create shard sizes between 20-30 GB per shard:
\*/15 \* \* \* \* curl -XPOST '[http://localhost:9200/yourindexaliasname/\_rollover](http://localhost:9200/yourindexaliasname/_rollover)' -d '{ "conditions" : { "max\_size": "125gb" }}' -k
If you have 5 shards and each shard shall be 25GB in Size
If you can't shrink the old data reindex it and get rid of too many indices!
Nice to read and get a better understanding on scaling your cluster.
[https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/sizing-domains.html](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/sizing-domains.html)
[https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/petabyte-scale.html](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/petabyte-scale.html)
--- RAM Storage Ratio ---
[https://www.elastic.co/guide/en/cloud-enterprise/1.1/ece-manage-node-configurations.html](https://www.elastic.co/guide/en/cloud-enterprise/1.1/ece-manage-node-configurations.html)
Good luck!
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [January 31, 2019, 10:43am UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488/5 "2019-01-31T10:43:55Z")
Given that you have a good amount of data coming in per day I think you might be better off right-sizing your shards from the outset and not have to go through using the shrink index API. Based on your heap size and amount of storage per node, aiming for an average shard size between 25GB and 30GB might be a good starting point.
---
### Author:  [@Amit\_Ripshtos](https://discuss.elastic.co/u/Amit_Ripshtos)
#### Post date: [January 31, 2019, 8:23pm UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488/6 "2019-01-31T20:23:00Z")
Thanks everyone, it was very helpful! I will read all the materials and will test everything.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 28, 2019, 8:23pm UTC](https://discuss.elastic.co/t/most-right-architecture-for-my-cluster/166488/7 "2019-02-28T20:23:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.