# Migration to ElasticSearch: Advices on how many shards, replicas and instance size **URL:** \ **Category:** Elasticsearch\ **Created:** [May 30, 2018, 3:06pm UTC](https://discuss.elastic.co/t/migration-to-elasticsearch-advices-on-how-many-shards-replicas-and-instance-size/133901 "2018-05-30T15:06:22Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ViniciusSPaiva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/viniciusspaiva/32/67297_2.png) [@ViniciusSPaiva](https://discuss.elastic.co/u/ViniciusSPaiva)\ **Post date:** [May 30, 2018, 3:06pm UTC](https://discuss.elastic.co/t/migration-to-elasticsearch-advices-on-how-many-shards-replicas-and-instance-size/133901/1 "2018-05-30T15:06:22Z")
Hello there! This is my first post on the forums and I'll be working with Elastic Search also for the first time, so bear with me a little 🙂 We have a cloud application that indexes and searches file contents (PDFs, Office, etc) and we currently use the AWS service, CloudSearch. The service is good, as we don't really need to configure anything other than the fields but the price was starting to become really expensive. So we decided to move to the Elastic Search service also provided by AWS. Our index size is approximately 30GB (ever growing) and the number of searches is currently small (a couple of dozens per day) but returns a lot of results (1000) without pagination. So the first thing that is different is that I need to decide the number of nodes, shards, replicas, master nodes, instance types... đŸ˜± I saw some articles talking about a size of 50GB per shard, and that too many on a small instance can be very ineffective. I was thinking of something like the setup below. Any advice would be extremely helpful, my biggest doubt is about the number of shards. - Instance type: t2.medium (2 vCPU, 4 GiB) - As the number of searches are small, I was also thinking about a t2.small (1 vCPU, 2 GiB) initially. - Two nodes (one master, one replica) - Two shards (Because of the small instances, I thought too many shards would be bad) - One replica (For security and availability) Thanks!! ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 30, 2018, 3:45pm UTC](https://discuss.elastic.co/t/migration-to-elasticsearch-advices-on-how-many-shards-replicas-and-instance-size/133901/2 "2018-05-30T15:45:03Z")
May I suggest you look at the following resources about sizing: [https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing) > **[How many shards should I have in my Elasticsearch cluster? | Elastic](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)** > > Elasticsearch is a very versatile platform, that supports a variety of use cases, and provides great flexibility around data organisation and replication strategies. This flexibility can however somet... > **[NetSecureDay: Managing your Black Friday Logs](https://speakerdeck.com/elastic/netsecureday-managing-your-black-friday-logs)** > > Surveiller une application complexe n’est pas une tĂąche aisĂ©e, mais avec les bons outils, ce n’est pas si sorcier. NĂ©anmoins, des pĂ©riodes fortes telles que les opĂ©rations de type « Black Friday » (Vendredi noir) ou pĂ©riode de NoĂ«l peuvent pousser... BTW did you look at [https://www.elastic.co/cloud](https://www.elastic.co/cloud) and [https://aws.amazon.com/marketplace/pp/B01N6YCISK](https://aws.amazon.com/marketplace/pp/B01N6YCISK) ? Cloud by elastic is the only way to have access to X-Pack. Think about what is there yet like Security, Monitoring, Reporting and what is coming like Canvas, SQL... ---
**Author:** ![ViniciusSPaiva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/viniciusspaiva/32/67297_2.png) [@ViniciusSPaiva](https://discuss.elastic.co/u/ViniciusSPaiva)\ **Post date:** [May 30, 2018, 5:36pm UTC](https://discuss.elastic.co/t/migration-to-elasticsearch-advices-on-how-many-shards-replicas-and-instance-size/133901/3 "2018-05-30T17:36:50Z")
I took a look on some articles, yes. But was still a little insecure, because you choose the quantity of shards and replicas in index creation and you can't change it afterwards, right? About the cloud from Elastic, I didn't know about it. But it has a problem similar to CloudSearch: the storage is linked with the node size. When I increase the storage, my cluster CPU and memory also increases. I would like to have the option to just increase the storage, because our index will grow indefinitely but our current use (searches) is small. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 1, 2018, 9:34am UTC](https://discuss.elastic.co/t/migration-to-elasticsearch-advices-on-how-many-shards-replicas-and-instance-size/133901/4 "2018-06-01T09:34:14Z")
> [@ViniciusSPaiva](#): > > because you choose the quantity of shards and replicas in index creation and you can't change it afterwards, right? We now have the [Shrink API](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/indices-shrink-index.html) and the [Split API](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/indices-split-index.html) though. > But it has a problem similar to CloudSearch: the storage is linked with the node size. When I increase the storage, my cluster CPU and memory also increases. This is going to change anytime soon. So you will be able in the near future to choose depending basically on your use case what kind of node you would prefer. I don't know the exact date for this though. ---
**Author:** ![Wayne\_Taylor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wayne_taylor/32/45984_2.png) [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Post date:** [June 1, 2018, 1:02pm UTC](https://discuss.elastic.co/t/migration-to-elasticsearch-advices-on-how-many-shards-replicas-and-instance-size/133901/5 "2018-06-01T13:02:35Z")
Hi there, Is that 30gb total as that seems tiny. I have some indexes that I ingest at 50gb per month and rotate them in that manner so index name is like index-yyyy-mm. I'd recommend you look at your volume and determine the ingest size per day before deciding on your index strategy. On elastic cloud I'd recommend we've used for over a year ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 29, 2018, 1:02pm UTC](https://discuss.elastic.co/t/migration-to-elasticsearch-advices-on-how-many-shards-replicas-and-instance-size/133901/6 "2018-06-29T13:02:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.