# Many small indices vs One large index **URL:** \ **Category:** Elasticsearch\ **Created:** [October 14, 2020, 8:28am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008 "2020-10-14T08:28:38Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![hemangr8](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hemangr8/32/77147_2.png) [@hemangr8](https://discuss.elastic.co/u/hemangr8)\ **Post date:** [October 14, 2020, 8:28am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008/1 "2020-10-14T08:28:38Z")
I know this question has been asked many times, but I needed more clarity on this topic. Let's say I have 5000 customer data which has the same number of fields in each document but is independent of each other, i.e. I will never be needing to query multiple customers together. Each customer will have 2 indices and that will make 10000 indices in total. I can combine all of them into 2 big indices with some tens of shards or keep them all in separate indices. I would also like to know about how many recommended nodes in each case would I need, considering that there should be no more than a thousand shards per node means that in many indices configuration I would at least need 10 nodes and in the huge index configuration I can keep 1 node but add if necessary. I was wondering if I could get some suggestions on what would be the best configuration. An early response will be appreciated. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 14, 2020, 8:33am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008/2 "2020-10-14T08:33:03Z")
Lots of small indices and shards is very inefficient, so it is almost always better to consolidate. ---
**Author:** ![hemangr8](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hemangr8/32/77147_2.png) [@hemangr8](https://discuss.elastic.co/u/hemangr8)\ **Post date:** [October 14, 2020, 8:34am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008/3 "2020-10-14T08:34:08Z")
Thanks for the early response. Could you please help me in the second part of the question as well? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 14, 2020, 8:37am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008/4 "2020-10-14T08:37:50Z")
In order to have a highly available cluster you need at least 3 nodes. As you do not query across users I would recommend looking at using [routing](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/mapping-routing-field.html) when indexing and querying. How many shards you need depend on the data volume and work load. ---
**Author:** ![hemangr8](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hemangr8/32/77147_2.png) [@hemangr8](https://discuss.elastic.co/u/hemangr8)\ **Post date:** [October 14, 2020, 8:48am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008/5 "2020-10-14T08:48:28Z")
Thanks routing seems the way to go instead of filtering based on user ID. So my load for querying will be a few 100 qps at max load, for that I think 10 shards should be enough. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 14, 2020, 8:50am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008/6 "2020-10-14T08:50:49Z")
Sounds like a reasonable starting point. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 11, 2020, 8:50am UTC](https://discuss.elastic.co/t/many-small-indices-vs-one-large-index/252008/7 "2020-11-11T08:50:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.