# How to choose indexes

**URL:** <https://discuss.elastic.co/t/how-to-choose-indexes/13081>\
**Category:** Elasticsearch\
**Created:** [August 5, 2013, 10:50pm UTC](https://discuss.elastic.co/t/how-to-choose-indexes/13081 "2013-08-05T22:50:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![imanlists](https://avatars.discourse-cdn.com/v4/letter/i/8edcca/32.png) [@imanlists](https://discuss.elastic.co/u/imanlists)\
**Post date:** [August 5, 2013, 10:50pm UTC](https://discuss.elastic.co/t/how-to-choose-indexes/13081/1 "2013-08-05T22:50:47Z")

</div>

Is there an ideal amount of indexes to have and if so why? I am trying to  
figure out which indexes to store my types under and wanted to make sure I  
was considering all of the factors.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [August 6, 2013, 1:06am UTC](https://discuss.elastic.co/t/how-to-choose-indexes/13081/2 "2013-08-06T01:06:50Z")

</div>

A rule of thumb could be: number of shard per node equal to number of cores (processor).  
So if you have 3 nodes, 8 cores per node, you have 24 cores in your cluster.  
If you want 1 replica per shard, you can think that the number of primary shards in 12.

It could be 12 index with 1 shard or 1 index with 12 shards...

But it also really depends on your usage. If some shards are less used than others (think time based indices), you can have more shards on a box than the number of cores.

Ideally, you should test it on the targeted hardware to find _your_ magic numbers.

My 2 cents.

--  
David 😉  
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 août 2013 à 00:50, [imanlists@gmail.com](mailto:imanlists@gmail.com) a écrit :

## Is there an ideal amount of indexes to have and if so why? I am trying to figure out which indexes to store my types under and wanted to make sure I was considering all of the factors.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\
**Post date:** [August 6, 2013, 2:30am UTC](https://discuss.elastic.co/t/how-to-choose-indexes/13081/3 "2013-08-06T02:30:12Z")

</div>

Hehehe! My observations:

1. Elasticsearch Head is very popular, and a wonderful tool.
2. Elasticsearch Head's formatting is broken for 11 or more shards  
(double-digit shard numbers).

So my rule of thumb is a maximum of 10 shards. 😉

But seriously, I've had great success with 97M+ documents in 10 shards,  
whether in a 1-machine cluster (my laptop) or a 3-machine cluster (Solaris  
x86-64 VMs; shared CPUs I'm sure).

What I'm waiting for are the final fixes to the split-brain issues that  
some have seen. ES 0.90.0 is working fine in this environment so far,  
though.

On Monday, August 5, 2013 6:50:47 PM UTC-4, [iman...@gmail.com](mailto:iman...@gmail.com) wrote:

> Is there an ideal amount of indexes to have and if so why? I am trying to  
> figure out which indexes to store my types under and wanted to make sure I  
> was considering all of the factors.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [August 6, 2013, 7:53am UTC](https://discuss.elastic.co/t/how-to-choose-indexes/13081/4 "2013-08-06T07:53:45Z")

</div>

For the sake of completeness, you can over-allocate shards, so the number  
of indexes is no longer dependent on the hardware resources. No need to  
worry for an ideal number of indexes.

That is, you create one index over all nodes. Then you can configure index  
aliases on this index, with a tag field in the documents to filter out the  
docs of your alias. To the clients when using the ES API, it looks like you  
have many indexes (if you can post-process the \_index field in the response  
and enforce the tag field in the mapping). You can configure tens,  
hundreds, thousands of alias indexes, without having to worry about the  
organization of concrete indexes.

Still, there is a dependency, you should not let the shards volume grow too  
large. This depends on your performance requirements and what your node  
capacity is, how fast you want merging, recovering, moving shards etc.

kimchy on over-allocation of shards:  
[https://groups.google.com/forum/#!msg/elasticsearch/49q-\_AgQCp8/MRol0t9asEcJ](https://groups.google.com/forum/#!msg/elasticsearch/49q-_AgQCp8/MRol0t9asEcJ)

Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:22am UTC](https://discuss.elastic.co/t/how-to-choose-indexes/13081/5 "2017-07-06T02:22:44Z")

</div>


