# Index-Per-User Scale

**URL:** <https://discuss.elastic.co/t/index-per-user-scale/2519>\
**Category:** Elasticsearch\
**Created:** [June 11, 2015, 8:10pm UTC](https://discuss.elastic.co/t/index-per-user-scale/2519 "2015-06-11T20:10:19Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [June 11, 2015, 8:30pm UTC](https://discuss.elastic.co/t/index-per-user-scale/2519/2 "2015-06-11T20:30:31Z")

</div>

> [@daverus](#):
>
> If we use "index-per-user", the entire index is only 2-3MB for 10k documents (our documents are small, primarily non-analyzed fields)

This might be useful to you:

> <https://github.com/elastic/elasticsearch/issues/6720>
>
> Lucene has index-time sorting and early query termination capabilities. It would… be nice to integrate it into Elasticsearch in order to speed up queries whose sort order matches the index order. This \[presentation\](https://speakerdeck.com/elasticsearch/index-sorting-with-lucene) contains some information about these features and their implementation in Lucene.

Its obviously not done, but it could help fix these locality issues.

> [@daverus](#):
>
> - How much memory does each index take in the cluster state. Does it grow linearly?

I don't have a good idea how much memory each index takes but it certainly does grow linearly. Some of the

> [@daverus](#):
>
> - Is it crazy to attempt a cluster with \>1M indexes (mostly closed) and ~5k open indexes? If so, please explain why.

You'll probably be the only person doing it so you are likely to hit new and fun bugs no one has thought of. There are folks who have lots of always open indexes - I have two thousand or so - and I can feel cluster state actions being a bit sluggish. I'm still on an old version of Elasticsearch at the moment and I know that some of that has been fixed but I still expect the scale of what you are describing to be "fun".

I think its worth writing a test application that pretends to do what you are going to do - it could be as simple as a bash script that creates you million indexes, jams 20 docs in each, and closes them. Measuring memory consumption in Java is **hard** but you should be able to try and simulate your task and test it. I'd advise against doing what you plan to do without first proving it out that way.

Or go comment on the issue I linked above and maybe pitch in there somehow.

---

_[View the full topic](https://discuss.elastic.co/t/index-per-user-scale/2519)._