# Multiple indices vs. routing
**URL:** https://discuss.elastic.co/t/multiple-indices-vs-routing/10358
**Category:** Elasticsearch
**Created:** [January 15, 2013, 8:07pm UTC](https://discuss.elastic.co/t/multiple-indices-vs-routing/10358 "2013-01-15T20:07:55Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Michael\_Jackson](https://discuss.elastic.co/u/Michael_Jackson)
#### Post date: [January 15, 2013, 8:07pm UTC](https://discuss.elastic.co/t/multiple-indices-vs-routing/10358/1 "2013-01-15T20:07:55Z")
I'm building an app where I expect to index hundreds (maybe thousands) of
JSON documents every day per user. My question is about the trade-offs
between using an individual index per user and using the same index with a
different routing value per user.
Using the same index with the user id as a routing value is the simpler
solution, and seems like it will require less overhead because I won't be
creating a separate index for each user. However, it also means that my
single index will get very big and that may pose some problems as well. The
README suggests that using separate indices might be a better idea when
you're dealing with a large amount of data (see
[https://github.com/elasticsearch/elasticsearch](https://github.com/elasticsearch/elasticsearch) under the heading "Multi
Tenant – Indices and Types").
At what point does index size become such a problem that it makes sense to
have an index per user?
--
Michael Jackson
@mjackson
--
---
### Author:  [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)
#### Post date: [January 15, 2013, 11:12pm UTC](https://discuss.elastic.co/t/multiple-indices-vs-routing/10358/2 "2013-01-15T23:12:11Z")
How many users are we talking about?
On Tuesday, January 15, 2013 3:07:55 PM UTC-5, Michael Jackson wrote:
> I'm building an app where I expect to index hundreds (maybe thousands) of
> JSON documents every day per user. My question is about the trade-offs
> between using an individual index per user and using the same index with a
> different routing value per user.
>
> Using the same index with the user id as a routing value is the simpler
> solution, and seems like it will require less overhead because I won't be
> creating a separate index for each user. However, it also means that my
> single index will get very big and that may pose some problems as well. The
> README suggests that using separate indices might be a better idea when
> you're dealing with a large amount of data (see
> [GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine](https://github.com/elasticsearch/elasticsearch) under the heading "Multi
> Tenant – Indices and Types").
>
> At what point does index size become such a problem that it makes sense to
> have an index per user?
>
> --
> Michael Jackson
> @mjackson
--
---
### Author:  [@Michael\_Jackson](https://discuss.elastic.co/u/Michael_Jackson)
#### Post date: [January 15, 2013, 11:59pm UTC](https://discuss.elastic.co/t/multiple-indices-vs-routing/10358/3 "2013-01-15T23:59:54Z")
At this point not many. But we're hoping it will grow as large as it can. 🙂
A third option might be to start with just a single index with the
understanding that indexes cannot be modified once they are created. If we
do grow to the point where the single index is running too slowly we could
create a separate cluster that uses one index per user. Since we're using
the CouchDB river, it should be fairly easy to setup and modify all search
queries to point to the new cluster and indexes once it's ready.
--
Michael Jackson
@mjackson
On Tue, Jan 15, 2013 at 3:12 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> How many users are we talking about?
>
> On Tuesday, January 15, 2013 3:07:55 PM UTC-5, Michael Jackson wrote:
>
> > I'm building an app where I expect to index hundreds (maybe thousands) of
> > JSON documents every day per user. My question is about the trade-offs
> > between using an individual index per user and using the same index with a
> > different routing value per user.
> >
> > Using the same index with the user id as a routing value is the simpler
> > solution, and seems like it will require less overhead because I won't be
> > creating a separate index for each user. However, it also means that my
> > single index will get very big and that may pose some problems as well. The
> > README suggests that using separate indices might be a better idea when
> > you're dealing with a large amount of data (see [https://github.com/](https://github.com/)\*\*
> > elasticsearch/elasticsearch[https://github.com/elasticsearch/elasticsearch](https://github.com/elasticsearch/elasticsearch)under the heading "Multi Tenant – Indices and Types").
> >
> > At what point does index size become such a problem that it makes sense
> > to have an index per user?
> >
> > --
> > Michael Jackson
> > @mjackson
>
> --
--
---
### Author:  [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)
#### Post date: [January 16, 2013, 12:33am UTC](https://discuss.elastic.co/t/multiple-indices-vs-routing/10358/4 "2013-01-16T00:33:14Z")
These are all viable options in different settings. Personally, I would
approach it this way. If most of the searches are done against a single
user index and I can afford it (I don't mind overhead and I don't expect to
have more than a few thousand users) I would go with one index per user. It
provides a nice separation, fast queries, users don't mess each others
IDFs, etc. However, if I know that my application will grow above a few
thousand users or if searches are done against entire index I would go with
some other strategy. I would consider partitioning by time or create
separate indices for large users and combine several small users into a
single index.
Have you seen this presentation by Shay
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
? He is discussing this particular issue starting at 13:40.
On Tuesday, January 15, 2013 6:59:54 PM UTC-5, Michael Jackson wrote:
> At this point not many. But we're hoping it will grow as large as it can.
> 🙂
>
> A third option might be to start with just a single index with the
> understanding that indexes cannot be modified once they are created. If we
> do grow to the point where the single index is running too slowly we could
> create a separate cluster that uses one index per user. Since we're using
> the CouchDB river, it should be fairly easy to setup and modify all search
> queries to point to the new cluster and indexes once it's ready.
>
> --
> Michael Jackson
> @mjackson
>
> On Tue, Jan 15, 2013 at 3:12 PM, Igor Motov \<[imo...@gmail.com](mailto:imo...@gmail.com)\
>
> > wrote:
>
> > How many users are we talking about?
> >
> > On Tuesday, January 15, 2013 3:07:55 PM UTC-5, Michael Jackson wrote:
> >
> > > I'm building an app where I expect to index hundreds (maybe thousands)
> > > of JSON documents every day per user. My question is about the trade-offs
> > > between using an individual index per user and using the same index with a
> > > different routing value per user.
> > >
> > > Using the same index with the user id as a routing value is the simpler
> > > solution, and seems like it will require less overhead because I won't be
> > > creating a separate index for each user. However, it also means that my
> > > single index will get very big and that may pose some problems as well. The
> > > README suggests that using separate indices might be a better idea when
> > > you're dealing with a large amount of data (see [https://github.com/](https://github.com/)\*\*
> > > elasticsearch/elasticsearch[https://github.com/elasticsearch/elasticsearch](https://github.com/elasticsearch/elasticsearch)under the heading "Multi Tenant – Indices and Types").
> > >
> > > At what point does index size become such a problem that it makes sense
> > > to have an index per user?
> > >
> > > --
> > > Michael Jackson
> > > @mjackson
> >
> > --
--
---
### Author:  [@Michael\_Jackson](https://discuss.elastic.co/u/Michael_Jackson)
#### Post date: [January 16, 2013, 1:06am UTC](https://discuss.elastic.co/t/multiple-indices-vs-routing/10358/5 "2013-01-16T01:06:17Z")
Thanks for the advice. That sounds about right.
No, I hadn't seen Shay's presentation but I will be sure and watch it
tonight. Thanks again!
--
Michael Jackson
@mjackson
On Tue, Jan 15, 2013 at 4:33 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> These are all viable options in different settings. Personally, I would
> approach it this way. If most of the searches are done against a single
> user index and I can afford it (I don't mind overhead and I don't expect to
> have more than a few thousand users) I would go with one index per user. It
> provides a nice separation, fast queries, users don't mess each others
> IDFs, etc. However, if I know that my application will grow above a few
> thousand users or if searches are done against entire index I would go with
> some other strategy. I would consider partitioning by time or create
> separate indices for large users and combine several small users into a
> single index.
>
> Have you seen this presentation by Shay
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/videos/2012/06/05/big-data-search-and-analytics.html)? He is discussing this particular issue starting at 13:40.
>
> On Tuesday, January 15, 2013 6:59:54 PM UTC-5, Michael Jackson wrote:
>
> > At this point not many. But we're hoping it will grow as large as it can.
> > 🙂
> >
> > A third option might be to start with just a single index with the
> > understanding that indexes cannot be modified once they are created. If we
> > do grow to the point where the single index is running too slowly we could
> > create a separate cluster that uses one index per user. Since we're using
> > the CouchDB river, it should be fairly easy to setup and modify all search
> > queries to point to the new cluster and indexes once it's ready.
> >
> > --
> > Michael Jackson
> > @mjackson
> >
> > On Tue, Jan 15, 2013 at 3:12 PM, Igor Motov [imo...@gmail.com](mailto:imo...@gmail.com) wrote:
> >
> > > How many users are we talking about?
> > >
> > > On Tuesday, January 15, 2013 3:07:55 PM UTC-5, Michael Jackson wrote:
> > >
> > > > I'm building an app where I expect to index hundreds (maybe thousands)
> > > > of JSON documents every day per user. My question is about the trade-offs
> > > > between using an individual index per user and using the same index with a
> > > > different routing value per user.
> > > >
> > > > Using the same index with the user id as a routing value is the simpler
> > > > solution, and seems like it will require less overhead because I won't be
> > > > creating a separate index for each user. However, it also means that my
> > > > single index will get very big and that may pose some problems as well. The
> > > > README suggests that using separate indices might be a better idea when
> > > > you're dealing with a large amount of data (see [https://github.com/](https://github.com/)\*\*
> > > > elasti\*\*csearch/elasticsearch[https://github.com/elasticsearch/elasticsearch](https://github.com/elasticsearch/elasticsearch)under the heading "Multi Tenant – Indices and Types").
> > > >
> > > > At what point does index size become such a problem that it makes sense
> > > > to have an index per user?
> > > >
> > > > --
> > > > Michael Jackson
> > > > @mjackson
> > >
> > > --
> >
> > --
--
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:56am UTC](https://discuss.elastic.co/t/multiple-indices-vs-routing/10358/6 "2017-07-06T02:56:15Z")