# Balance between number of indices and shards per index
**URL:** https://discuss.elastic.co/t/balance-between-number-of-indices-and-shards-per-index/18520
**Category:** Elasticsearch
**Created:** [July 8, 2014, 7:16am UTC](https://discuss.elastic.co/t/balance-between-number-of-indices-and-shards-per-index/18520 "2014-07-08T07:16:56Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 8, 2014, 7:16am UTC](https://discuss.elastic.co/t/balance-between-number-of-indices-and-shards-per-index/18520/1 "2014-07-08T07:16:56Z")
I'm setting up an Elasticsearch-based log cluster and I'm having some
doubts about how I should choose the number of indices and shards.
By default, Logstash and Kibana use per-day indices and Elasticsearch
defaults to five shards per index. I'm worried that this will create
an excessive number of shards with a log retension of, say, 100 days.
With one replica per shard I'd be facing 1000 shards cluster-wide.
With three or four data nodes that's at least 250 shards per node.
Whether this is too much obviously depends on the node and perhaps
on the size of the daily indices, but regardless it doesn't seem
particularly advantageous with such a number of shards. Would it
make more sense to use week-based indices or reduce the number of
(primary) shards per index to two or three to get the number of
shards per node down towards or below 100? Or should I stop worrying?
--
Magnus Bäck | Software Engineer, Development Tools
[magnus.back@sonymobile.com](mailto:magnus.back@sonymobile.com) | Sony Mobile Communications
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/20140708071656.GA26839%40seldlx20533.corpusers.net](https://groups.google.com/d/msgid/elasticsearch/20140708071656.GA26839%40seldlx20533.corpusers.net).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 8, 2014, 7:33am UTC](https://discuss.elastic.co/t/balance-between-number-of-indices-and-shards-per-index/18520/2 "2014-07-08T07:33:44Z")
Short - Stop worrying!
Long - As you mentioned this is very dependant on your node specs, but
ideally you want one shard per node. However you can over-allocate and not
run into problems, plus it allows easier balancing when you add more nodes
to the cluster.
_U_sing daily is a better as you can drop smaller units (ie indexes) of
data to suit your needs, and if you use elasticsearch-curator then your
retention can be automatically managed.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)
web: [www.campaignmonitor.com](http://www.campaignmonitor.com)
On 8 July 2014 17:16, Magnus Bäck [magnus.back@sonymobile.com](mailto:magnus.back@sonymobile.com) wrote:
> I'm setting up an Elasticsearch-based log cluster and I'm having some
> doubts about how I should choose the number of indices and shards.
> By default, Logstash and Kibana use per-day indices and Elasticsearch
> defaults to five shards per index. I'm worried that this will create
> an excessive number of shards with a log retension of, say, 100 days.
> With one replica per shard I'd be facing 1000 shards cluster-wide.
> With three or four data nodes that's at least 250 shards per node.
>
> Whether this is too much obviously depends on the node and perhaps
> on the size of the daily indices, but regardless it doesn't seem
> particularly advantageous with such a number of shards. Would it
> make more sense to use week-based indices or reduce the number of
> (primary) shards per index to two or three to get the number of
> shards per node down towards or below 100? Or should I stop worrying?
>
> --
> Magnus Bäck | Software Engineer, Development Tools
> [magnus.back@sonymobile.com](mailto:magnus.back@sonymobile.com) | Sony Mobile Communications
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/20140708071656.GA26839%40seldlx20533.corpusers.net](https://groups.google.com/d/msgid/elasticsearch/20140708071656.GA26839%40seldlx20533.corpusers.net)
> .
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624YojpvL%3DXXXVhirUOg0Y2QmQ0MnnByWgbcEPi-i5NHegg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YojpvL%3DXXXVhirUOg0Y2QmQ0MnnByWgbcEPi-i5NHegg%40mail.gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:17am UTC](https://discuss.elastic.co/t/balance-between-number-of-indices-and-shards-per-index/18520/3 "2017-07-06T01:17:24Z")