# Large number of indexes

**URL:** <https://discuss.elastic.co/t/large-number-of-indexes/28853>\
**Category:** Elasticsearch\
**Created:** [September 8, 2015, 4:16pm UTC](https://discuss.elastic.co/t/large-number-of-indexes/28853 "2015-09-08T16:16:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Galim\_Kaudinov](https://avatars.discourse-cdn.com/v4/letter/g/e5b9ba/32.png) [@Galim\_Kaudinov](https://discuss.elastic.co/u/Galim_Kaudinov)\
**Post date:** [September 8, 2015, 4:16pm UTC](https://discuss.elastic.co/t/large-number-of-indexes/28853/1 "2015-09-08T16:16:38Z")

</div>

In my multi tenant application every tenant has an ability to create a custom data structure that is composed of different fields such as strings, bools, ints and etc. I am planning to create an index per data structure. So potentially there can be 20-30K user defined indexes and each index can contain 1K-100K documents.

Do you think this will work? or should I consider doing something different?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 8, 2015, 6:42pm UTC](https://discuss.elastic.co/t/large-number-of-indexes/28853/2 "2015-09-08T18:42:23Z")

</div>

Each shard in Elasticsearch is a Lucene instance, which uses a certain amount of file handles, memory and CPU cycles behind the scenes. Shards are therefore not for free, and having a very large number of small indices is generally a bad idea as it does not scale well. Discussions on this topic can be found [in this previous question](https://discuss.elastic.co/t/multi-tenancy-performance/3732) as well as [in this blog post](https://www.elastic.co/blog/found-sizing-elasticsearch#index-per-user).

---

<div class="post-metadata">

**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\
**Post date:** [September 9, 2015, 4:17am UTC](https://discuss.elastic.co/t/large-number-of-indexes/28853/3 "2015-09-09T04:17:54Z")

</div>

Maybe fewer indices and document routing would be better. Lots of shards, lots of problems.

## Otis

Monitoring \* Alerting \* Anomaly Detection \* Centralized Log Management  
Elasticsearch Consulting & Support \* [http://sematext.com/](http://sematext.com/)

---

<div class="post-metadata">

**Author:** ![Galim\_Kaudinov](https://avatars.discourse-cdn.com/v4/letter/g/e5b9ba/32.png) [@Galim\_Kaudinov](https://discuss.elastic.co/u/Galim_Kaudinov)\
**Post date:** [September 9, 2015, 4:48pm UTC](https://discuss.elastic.co/t/large-number-of-indexes/28853/4 "2015-09-09T16:48:54Z")

</div>

Thanks for the responses.

Do think it is possible to achieve this type of customization using "Dynamic Mapping"? I can create a dynamic mapping that will match based on the custom field name.

Another option is to create a one big index that will contain all possible custom field types \* 5. This solution appeals because of its simplicity, but than it will limit users to maximum number of fields in their data structure.

The end result is to make this user defined and populated data easily readable, searchable, sortable and etc...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:51pm UTC](https://discuss.elastic.co/t/large-number-of-indexes/28853/5 "2017-07-05T23:51:30Z")

</div>


