# AWS Elasticsearch Best Practice for Indexes

**URL:** <https://discuss.elastic.co/t/aws-elasticsearch-best-practice-for-indexes/208634>\
**Category:** Elasticsearch\
**Created:** [November 20, 2019, 7:10am UTC](https://discuss.elastic.co/t/aws-elasticsearch-best-practice-for-indexes/208634 "2019-11-20T07:10:28Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fei\_YU](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fei_yu/32/58002_2.png) [@Fei\_YU](https://discuss.elastic.co/u/Fei_YU)\
**Post date:** [November 20, 2019, 7:10am UTC](https://discuss.elastic.co/t/aws-elasticsearch-best-practice-for-indexes/208634/1 "2019-11-20T07:10:28Z")

</div>

We have been using AWS Elasticsearch Service for a while ( few years ) with the current configuration:

1. No Master dedicated Nodes.
2. 2k+ indexes and 10K primary shards, 20K shards with one replica.
3. 4 instances 8GB memory data node.
4. the total data size is about 40GB.
5. We were still running ES2.3, which is a very old version already.

Things worked smoothly until AWS decided to run maintenance and it would bring our cluster up and down for days, never finishing the migration and lowering the availability.

I have few questions:

1. I saw a lot of posts saying that as long as each shard is under 30GB, it is fine. But due to some business logic, we have to create a lot of indices and hence a lot of small shards. Would that impose performance penalties? What is a good strategy?
2. Should we upgrade to higher version of ES given the current situation as long as they enjoys better API functions?
3. Any other general advice?

Thank you so much in advance!

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 20, 2019, 9:19am UTC](https://discuss.elastic.co/t/aws-elasticsearch-best-practice-for-indexes/208634/2 "2019-11-20T09:19:07Z")

</div>

Upgrading is recommended. Also have a look at [this blog post about sharding](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 20, 2019, 11:43am UTC](https://discuss.elastic.co/t/aws-elasticsearch-best-practice-for-indexes/208634/3 "2019-11-20T11:43:28Z")

</div>

Welcome!

> [@Fei\_YU](#):
>
> We have been using AWS Elasticsearch Service

Not related to your question but did you look at [Elastic Cloud: Hosted Elasticsearch, Hosted Search | Elastic](https://www.elastic.co/cloud) and [https://aws.amazon.com/marketplace/pp/B01N6YCISK](https://aws.amazon.com/marketplace/pp/B01N6YCISK) ?

Cloud by elastic is one way to have access to all features, all managed by us. Think about what is there yet like Security, Monitoring, Reporting, SQL, Canvas, APM, Logs UI, Infra UI, SIEM, Maps UI and what is coming next 🙂 ...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 18, 2019, 11:43am UTC](https://discuss.elastic.co/t/aws-elasticsearch-best-practice-for-indexes/208634/4 "2019-12-18T11:43:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.