# Capacity Planning with ElasticSearch **URL:** \ **Category:** Elasticsearch\ **Created:** [April 22, 2014, 3:10am UTC](https://discuss.elastic.co/t/capacity-planning-with-elasticsearch/17117 "2014-04-22T03:10:26Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Mohannad\_Saeed](https://avatars.discourse-cdn.com/v4/letter/m/e5b9ba/32.png) [@Mohannad\_Saeed](https://discuss.elastic.co/u/Mohannad_Saeed)\ **Post date:** [April 22, 2014, 3:10am UTC](https://discuss.elastic.co/t/capacity-planning-with-elasticsearch/17117/1 "2014-04-22T03:10:26Z")
Greetings all, Few questions to help us in the capacity planning: 1. What is the memory footprint of an empty ElasticSearch index consisting of a single shard and zero replicas? 2. Is there a difference in footprint between 1 empty index with 1000 shards and 1000 empty indexes each with 1 shard (both with 0 replicas)? 3. Is it practical to create an ElasticSearch index for each user, if you have millions of them, each with thousands to few million small JSON documents? 4. Does ElasticSearch require an index shard to be fully (not paged) loaded into memory during querying? 5. Is using types in a single index different (in query performance) than using multiple indexes per type? Kind regards -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/72a5f036-c039-43ba-a2ab-c1b84e3c47ab%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/72a5f036-c039-43ba-a2ab-c1b84e3c47ab%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mohannad\_Saeed](https://avatars.discourse-cdn.com/v4/letter/m/e5b9ba/32.png) [@Mohannad\_Saeed](https://discuss.elastic.co/u/Mohannad_Saeed)\ **Post date:** [April 23, 2014, 3:04am UTC](https://discuss.elastic.co/t/capacity-planning-with-elasticsearch/17117/2 "2014-04-23T03:04:04Z")
Any experienced dude with ES to answer this?? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3e238739-2928-4aed-9883-094452bc6c11%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e238739-2928-4aed-9883-094452bc6c11%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 23, 2014, 4:07am UTC](https://discuss.elastic.co/t/capacity-planning-with-elasticsearch/17117/3 "2014-04-23T04:07:58Z")
1 and 2 - It'd probably be easiest to try this yourself 🙂 3 - not really, you should look into routing. 4 - only the index metadata is stored in memory. However doing aggregations will pull the applicable data into memory. 5 - not sure. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 23 April 2014 13:04, Mohannad Saeed [geekox86@gmail.com](mailto:geekox86@gmail.com) wrote: > Any experienced dude with ES to answer this?? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/3e238739-2928-4aed-9883-094452bc6c11%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e238739-2928-4aed-9883-094452bc6c11%40googlegroups.com) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624bXJieGyw2mJccr8ytNLMBRqV1sCUQO2j5ChG8n\_9O%2Bjw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624bXJieGyw2mJccr8ytNLMBRqV1sCUQO2j5ChG8n_9O%2Bjw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Tim\_Uckun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_uckun/32/1256_2.png) [@Tim\_Uckun](https://discuss.elastic.co/u/Tim_Uckun)\ **Post date:** [April 23, 2014, 9:39am UTC](https://discuss.elastic.co/t/capacity-planning-with-elasticsearch/17117/4 "2014-04-23T09:39:07Z")
To follow up on this... As a general rule is it better to have one horse size index or a hundred duck sized indices. I am thinking about those types of searches where you might frequently search a subset of the data. For example keeping a separate index for every customer because normally the app restricts itself to only dealing with one customer at a time. Perhaps doing a compound split based on customer and year if your searches rarely go outside of the current year. Thanks. On Wednesday, April 23, 2014 4:07:58 PM UTC+12, Mark Walkom wrote: > 1 and 2 - It'd probably be easiest to try this yourself 🙂 > 3 - not really, you should look into routing. > 4 - only the index metadata is stored in memory. However doing > aggregations will pull the applicable data into memory. > 5 - not sure. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \ > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 23 April 2014 13:04, Mohannad Saeed \<[geek...@gmail.com](mailto:geek...@gmail.com) \\>wrote: > > > Any experienced dude with ES to answer this?? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/3e238739-2928-4aed-9883-094452bc6c11%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3e238739-2928-4aed-9883-094452bc6c11%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/28d1e12a-a381-4921-b13e-83640767e281%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/28d1e12a-a381-4921-b13e-83640767e281%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 23, 2014, 11:01am UTC](https://discuss.elastic.co/t/capacity-planning-with-elasticsearch/17117/5 "2014-04-23T11:01:14Z")
It depends - on your data set, your queries, your cluster specs.Having tens to hundreds of thousands (or millions) of indexes will have a performance impact that will only increase with numbers, so the lower you can keep it though planning the better. But to counter that, the bigger your indexes, the longer it will take to query and you have a reduced agility to manipulate said indexes 😉 Which is why the answer to a lot of this sort of thing is - it depends. As an example of planning, it might be better to think ahead if you are aiming for such large sizes and give your app the ability to talk to multiple clusters, which will allow you to move customers into high/low performance/capacity clusters. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 23 April 2014 19:39, Tim Uckun [timuckun@gmail.com](mailto:timuckun@gmail.com) wrote: > To follow up on this... > > As a general rule is it better to have one horse size index or a hundred > duck sized indices. I am thinking about those types of searches where you > might frequently search a subset of the data. For example keeping a > separate index for every customer because normally the app restricts itself > to only dealing with one customer at a time. Perhaps doing a compound > split based on customer and year if your searches rarely go outside of the > current year. > > Thanks. > > On Wednesday, April 23, 2014 4:07:58 PM UTC+12, Mark Walkom wrote: > > > 1 and 2 - It'd probably be easiest to try this yourself 🙂 > > 3 - not really, you should look into routing. > > 4 - only the index metadata is stored in memory. However doing > > aggregations will pull the applicable data into memory. > > 5 - not sure. > > > > Regards, > > Mark Walkom > > > > Infrastructure Engineer > > Campaign Monitor > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > On 23 April 2014 13:04, Mohannad Saeed [geek...@gmail.com](mailto:geek...@gmail.com) wrote: > > > > > Any experienced dude with ES to answer this?? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/3e238739-2928-4aed-9883-094452bc6c11% > > > [40googlegroups.com](http://40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/28d1e12a-a381-4921-b13e-83640767e281%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/28d1e12a-a381-4921-b13e-83640767e281%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/28d1e12a-a381-4921-b13e-83640767e281%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/28d1e12a-a381-4921-b13e-83640767e281%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624Z6DbinW8Z%3Dr6RACqHSBnYCOwv245VsSQ4Wi4F-GSkZ\_A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Z6DbinW8Z%3Dr6RACqHSBnYCOwv245VsSQ4Wi4F-GSkZ_A%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:34am UTC](https://discuss.elastic.co/t/capacity-planning-with-elasticsearch/17117/6 "2017-07-06T01:34:08Z")