# Optimizing number of shards
**URL:** https://discuss.elastic.co/t/optimizing-number-of-shards/4830
**Category:** Elasticsearch
**Created:** [July 10, 2011, 7:27pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830 "2011-07-10T19:27:32Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)
#### Post date: [July 10, 2011, 7:27pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/1 "2011-07-10T19:27:32Z")
We are indexing about 3m documents / day (20kb/doc), for a total of
90-days with 1-replica. Each day corresponds to a new index, thereby
creating 90 indexes. In addition, each index uses a custom mapping,
this mapping is the same for all 90-indexes.
Given the above scenario, what would be an optimum number of shards
and any other setting that should be considered for a 5-node ES setup?
-anurag
---
### Author:  [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [July 10, 2011, 7:31pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/2 "2011-07-10T19:31:43Z")
I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
already a high load ;)).
But check if indexing speed is enough...
On 10 Jul., 21:27, Anurag [anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) wrote:
> We are indexing about 3m documents / day (20kb/doc), for a total of
> 90-days with 1-replica. Each day corresponds to a new index, thereby
> creating 90 indexes. In addition, each index uses a custom mapping,
> this mapping is the same for all 90-indexes.
>
> Given the above scenario, what would be an optimum number of shards
> and any other setting that should be considered for a 5-node ES setup?
>
> -anurag
---
### Author:  [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)
#### Post date: [July 10, 2011, 7:43pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/3 "2011-07-10T19:43:06Z")
I don't see any latency while its getting indexed, so I guess that's okay.
On same note, is there a way to specify default # of shards for the
entire cluster?
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
allows one to specify for a given index, but I don't see a default
systemwide setting as it exists for mapping:
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
-anurag
On Sun, Jul 10, 2011 at 12:31 PM, Karussell
[tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) wrote:
> I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> already a high load ;)).
>
> But check if indexing speed is enough...
>
> On 10 Jul., 21:27, Anurag [anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) wrote:
>
> > We are indexing about 3m documents / day (20kb/doc), for a total of
> > 90-days with 1-replica. Each day corresponds to a new index, thereby
> > creating 90 indexes. In addition, each index uses a custom mapping,
> > this mapping is the same for all 90-indexes.
> >
> > Given the above scenario, what would be an optimum number of shards
> > and any other setting that should be considered for a 5-node ES setup?
> >
> > -anurag
---
### Author:  [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)
#### Post date: [July 10, 2011, 7:47pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/4 "2011-07-10T19:47:08Z")
nm, found it:
"index" : {
"number\_of\_shards":1,
"number\_of\_replicas":1,
}
}
On Sun, Jul 10, 2011 at 12:43 PM, Anurag [anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) wrote:
> I don't see any latency while its getting indexed, so I guess that's okay.
>
> On same note, is there a way to specify default # of shards for the
> entire cluster?
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
> allows one to specify for a given index, but I don't see a default
> systemwide setting as it exists for mapping:
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/dynamic-mapping.html)
>
> -anurag
>
> On Sun, Jul 10, 2011 at 12:31 PM, Karussell
> [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) wrote:
>
> > I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> > already a high load ;)).
> >
> > But check if indexing speed is enough...
> >
> > On 10 Jul., 21:27, Anurag [anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) wrote:
> >
> > > We are indexing about 3m documents / day (20kb/doc), for a total of
> > > 90-days with 1-replica. Each day corresponds to a new index, thereby
> > > creating 90 indexes. In addition, each index uses a custom mapping,
> > > this mapping is the same for all 90-indexes.
> > >
> > > Given the above scenario, what would be an optimum number of shards
> > > and any other setting that should be considered for a 5-node ES setup?
> > >
> > > -anurag
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [July 10, 2011, 9:28pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/5 "2011-07-10T21:28:09Z")
Check index templates: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) to simplify the settings / mappings to automatically apply for this case.
On Sunday, July 10, 2011 at 10:47 PM, Anurag wrote:
> nm, found it:
> "index" : {
> "number\_of\_shards":1,
> "number\_of\_replicas":1,
> }
>
> }
>
> On Sun, Jul 10, 2011 at 12:43 PM, Anurag \<[anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) ([mailto:anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com))\> wrote:
>
> > I don't see any latency while its getting indexed, so I guess that's okay.
> >
> > On same note, is there a way to specify default # of shards for the
> > entire cluster?
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
> > allows one to specify for a given index, but I don't see a default
> > systemwide setting as it exists for mapping:
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/dynamic-mapping.html)
> >
> > -anurag
> >
> > On Sun, Jul 10, 2011 at 12:31 PM, Karussell
> > \<[tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) ([mailto:tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com))\> wrote:
> >
> > > I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> > > already a high load ;)).
> > >
> > > But check if indexing speed is enough...
> > >
> > > On 10 Jul., 21:27, Anurag \<[anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > >
> > > > We are indexing about 3m documents / day (20kb/doc), for a total of
> > > > 90-days with 1-replica. Each day corresponds to a new index, thereby
> > > > creating 90 indexes. In addition, each index uses a custom mapping,
> > > > this mapping is the same for all 90-indexes.
> > > >
> > > > Given the above scenario, what would be an optimum number of shards
> > > > and any other setting that should be considered for a 5-node ES setup?
> > > >
> > > > -anurag
---
### Author:  [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)
#### Post date: [July 10, 2011, 10:58pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/6 "2011-07-10T22:58:54Z")
Shay,
Does having a template affect performance? With the bulk index API, is
the template invoked everytime a document is indexed or is there some
sort of internal flag to only invoke template for new indices?
-anurag
On Sun, Jul 10, 2011 at 2:28 PM, Shay Banon
[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> Check index
> templates: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) to
> simplify the settings / mappings to automatically apply for this case.
>
> On Sunday, July 10, 2011 at 10:47 PM, Anurag wrote:
>
> nm, found it:
> "index" : {
> "number\_of\_shards":1,
> ""number\_of\_replicas":1,
> }
>
> }
>
> On Sun, Jul 10, 2011 at 12:43 PM, Anurag [anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) wrote:
>
> I don't see any latency while its getting indexed, so I guess that's okay.
>
> On same note, is there a way to specify default # of shards for the
> entire cluster?
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
> allows one to specify for a given index, but I don't see a default
> systemwide setting as it exists for mapping:
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/dynamic-mapping.html)
>
> -anurag
>
> On Sun, Jul 10, 2011 at 12:31 PM, Karussell
> [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) wrote:
>
> I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> already a high load ;)).
>
> But check if indexing speed is enough...
>
> On 10 Jul., 21:27, Anurag [anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) wrote:
>
> We are indexing about 3m documents / day (20kb/doc), for a total of
> 90-days with 1-replica. Each day corresponds to a new index, thereby
> creating 90 indexes. In addition, each index uses a custom mapping,
> this mapping is the same for all 90-indexes.
>
> Given the above scenario, what would be an optimum number of shards
> and any other setting that should be considered for a 5-node ES setup?
>
> -anurag
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [July 10, 2011, 11:04pm UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/7 "2011-07-10T23:04:47Z")
The index template is applied only when the index gets created.
On Monday, July 11, 2011 at 1:58 AM, Anurag wrote:
> Shay,
> Does having a template affect performance? With the bulk index API, is
> the template invoked everytime a document is indexed or is there some
> sort of internal flag to only invoke template for new indices?
>
> -anurag
>
> On Sun, Jul 10, 2011 at 2:28 PM, Shay Banon
> \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) ([mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com))\> wrote:
>
> > Check index
> > templates: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) to
> > simplify the settings / mappings to automatically apply for this case.
> >
> > On Sunday, July 10, 2011 at 10:47 PM, Anurag wrote:
> >
> > nm, found it:
> > "index" : {
> > "number\_of\_shards":1,
> > ""number\_of\_replicas":1,
> > }
> >
> > }
> >
> > On Sun, Jul 10, 2011 at 12:43 PM, Anurag \<[anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) ([mailto:anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com))\> wrote:
> >
> > I don't see any latency while its getting indexed, so I guess that's okay.
> >
> > On same note, is there a way to specify default # of shards for the
> > entire cluster?
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
> > allows one to specify for a given index, but I don't see a default
> > systemwide setting as it exists for mapping:
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/dynamic-mapping.html)
> >
> > -anurag
> >
> > On Sun, Jul 10, 2011 at 12:31 PM, Karussell
> > \<[tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) ([mailto:tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com))\> wrote:
> >
> > I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> > already a high load ;)).
> >
> > But check if indexing speed is enough...
> >
> > On 10 Jul., 21:27, Anurag \<[anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> >
> > We are indexing about 3m documents / day (20kb/doc), for a total of
> > 90-days with 1-replica. Each day corresponds to a new index, thereby
> > creating 90 indexes. In addition, each index uses a custom mapping,
> > this mapping is the same for all 90-indexes.
> >
> > Given the above scenario, what would be an optimum number of shards
> > and any other setting that should be considered for a 5-node ES setup?
> >
> > -anurag
---
### Author:  [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)
#### Post date: [July 11, 2011, 12:37am UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/8 "2011-07-11T00:37:03Z")
The following returns:
{"error":"ClassCastException[java.util.ArrayList cannot be cast to
java.util.Map]","status":500}
curl -XPUT localhost:9200/\_template/template\_1 -d '
{
"template" : "socorro\*",
"settings" : {
"number\_of\_shards" : 1
},
"mappings" : {
"date\_formats": [
"yyyy-MM-dd HH:mm:ss.SSSSSS"
],
"properties": {
"completeddatetime": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"date\_processed": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"client\_crash\_date": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"build\_date": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"startedDateTime": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"signature": {
"type": "multi\_field",
"fields": {
"signature": {
"type": "string",
"index": "analyzed"
},
"full": {
"type": "string",
"index": "not\_analyzed"
}
}
}
}
}
}
'
On Sun, Jul 10, 2011 at 4:04 PM, Shay Banon
[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> The index template is applied only when the index gets created.
>
> On Monday, July 11, 2011 at 1:58 AM, Anurag wrote:
>
> Shay,
> Does having a template affect performance? With the bulk index API, is
> the template invoked everytime a document is indexed or is there some
> sort of internal flag to only invoke template for new indices?
>
> -anurag
>
> On Sun, Jul 10, 2011 at 2:28 PM, Shay Banon
> [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
>
> Check index
> templates: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) to
> simplify the settings / mappings to automatically apply for this case.
>
> On Sunday, July 10, 2011 at 10:47 PM, Anurag wrote:
>
> nm, found it:
> "index" : {
> "number\_of\_shards":1,
> ""number\_of\_replicas":1,
> }
>
> }
>
> On Sun, Jul 10, 2011 at 12:43 PM, Anurag [anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) wrote:
>
> I don't see any latency while its getting indexed, so I guess that's okay.
>
> On same note, is there a way to specify default # of shards for the
> entire cluster?
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
> allows one to specify for a given index, but I don't see a default
> systemwide setting as it exists for mapping:
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/dynamic-mapping.html)
>
> -anurag
>
> On Sun, Jul 10, 2011 at 12:31 PM, Karussell
> [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) wrote:
>
> I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> already a high load ;)).
>
> But check if indexing speed is enough...
>
> On 10 Jul., 21:27, Anurag [anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) wrote:
>
> We are indexing about 3m documents / day (20kb/doc), for a total of
> 90-days with 1-replica. Each day corresponds to a new index, thereby
> creating 90 indexes. In addition, each index uses a custom mapping,
> this mapping is the same for all 90-indexes.
>
> Given the above scenario, what would be an optimum number of shards
> and any other setting that should be considered for a 5-node ES setup?
>
> -anurag
---
### Author:  [@Anurag\_Phadke](https://discuss.elastic.co/u/Anurag_Phadke)
#### Post date: [July 11, 2011, 12:41am UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/9 "2011-07-11T00:41:13Z")
nm, didn't realize it needs a type, the following works:
curl -XPUT localhost:9200/\_template/template\_1 -d '
{
"template" : "socorro\*",
"settings" : {
"number\_of\_shards" : 1
},
"mappings" : {
"crash\_reports": {
"date\_formats": [
"yyyy-MM-dd HH:mm:ss.SSSSSS"
],
"properties": {
"completeddatetime": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"date\_processed": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"client\_crash\_date": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"build\_date": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"startedDateTime": {
"format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
"type": "date"
},
"signature": {
"type": "multi\_field",
"fields": {
"signature": {
"type": "string",
"index": "analyzed"
},
"full": {
"type": "string",
"index": "not\_analyzed"
}
}
}
}
}
}
}
'
On Sun, Jul 10, 2011 at 5:37 PM, Anurag [anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) wrote:
> The following returns:
> {"error":"ClassCastException[java.util.ArrayList cannot be cast to
> java.util.Map]","status":500}
>
> curl -XPUT localhost:9200/\_template/template\_1 -d '
> {
> "template" : "socorro\*",
> "settings" : {
> "number\_of\_shards" : 1
> },
> "mappings" : {
> "date\_formats": [
> "yyyy-MM-dd HH:mm:ss.SSSSSS"
> ],
> "properties": {
> "completeddatetime": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "date\_processed": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "client\_crash\_date": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "build\_date": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "startedDateTime": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "signature": {
> "type": "multi\_field",
> "fields": {
> "signature": {
> "type": "string",
> "index": "analyzed"
> },
> "full": {
> "type": "string",
> "index": "not\_analyzed"
> }
> }
> }
> }
> }
> }
> '
>
> On Sun, Jul 10, 2011 at 4:04 PM, Shay Banon
> [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
>
> > The index template is applied only when the index gets created.
> >
> > On Monday, July 11, 2011 at 1:58 AM, Anurag wrote:
> >
> > Shay,
> > Does having a template affect performance? With the bulk index API, is
> > the template invoked everytime a document is indexed or is there some
> > sort of internal flag to only invoke template for new indices?
> >
> > -anurag
> >
> > On Sun, Jul 10, 2011 at 2:28 PM, Shay Banon
> > [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> >
> > Check index
> > templates: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) to
> > simplify the settings / mappings to automatically apply for this case.
> >
> > On Sunday, July 10, 2011 at 10:47 PM, Anurag wrote:
> >
> > nm, found it:
> > "index" : {
> > "number\_of\_shards":1,
> > ""number\_of\_replicas":1,
> > }
> >
> > }
> >
> > On Sun, Jul 10, 2011 at 12:43 PM, Anurag [anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) wrote:
> >
> > I don't see any latency while its getting indexed, so I guess that's okay.
> >
> > On same note, is there a way to specify default # of shards for the
> > entire cluster?
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
> > allows one to specify for a given index, but I don't see a default
> > systemwide setting as it exists for mapping:
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/dynamic-mapping.html)
> >
> > -anurag
> >
> > On Sun, Jul 10, 2011 at 12:31 PM, Karussell
> > [tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) wrote:
> >
> > I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> > already a high load ;)).
> >
> > But check if indexing speed is enough...
> >
> > On 10 Jul., 21:27, Anurag [anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) wrote:
> >
> > We are indexing about 3m documents / day (20kb/doc), for a total of
> > 90-days with 1-replica. Each day corresponds to a new index, thereby
> > creating 90 indexes. In addition, each index uses a custom mapping,
> > this mapping is the same for all 90-indexes.
> >
> > Given the above scenario, what would be an optimum number of shards
> > and any other setting that should be considered for a 5-node ES setup?
> >
> > -anurag
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [July 11, 2011, 1:10am UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/10 "2011-07-11T01:10:53Z")
Yuck!, what a fugly failure message, pushed a nicer one to master.
On Monday, July 11, 2011 at 3:41 AM, Anurag wrote:
> nm, didn't realize it needs a type, the following works:
>
> curl -XPUT localhost:9200/\_template/template\_1 -d '
> {
> "template" : "socorro\*",
> "settings" : {
> "number\_of\_shards" : 1
> },
> "mappings" : {
> "crash\_reports": {
> "date\_formats": [
> "yyyy-MM-dd HH:mm:ss.SSSSSS"
> ],
> "properties": {
> "completeddatetime": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "date\_processed": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "client\_crash\_date": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "build\_date": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "startedDateTime": {
> "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> "type": "date"
> },
> "signature": {
> "type": "multi\_field",
> "fields": {
> "signature": {
> "type": "string",
> "index": "analyzed"
> },
> "full": {
> "type": "string",
> "index": "not\_analyzed"
> }
> }
> }
> }
> }
> }
> }
> '
>
> On Sun, Jul 10, 2011 at 5:37 PM, Anurag \<[anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) ([mailto:anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com))\> wrote:
>
> > The following returns:
> > {"error":"ClassCastException[java.util.ArrayList cannot be cast to
> > java.util.Map]","status":500}
> >
> > curl -XPUT localhost:9200/\_template/template\_1 -d '
> > {
> > "template" : "socorro\*",
> > "settings" : {
> > "number\_of\_shards" : 1
> > },
> > "mappings" : {
> > "date\_formats": [
> > "yyyy-MM-dd HH:mm:ss.SSSSSS"
> > ],
> > "properties": {
> > "completeddatetime": {
> > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> > "type": "date"
> > },
> > "date\_processed": {
> > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> > "type": "date"
> > },
> > "client\_crash\_date": {
> > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> > "type": "date"
> > },
> > "build\_date": {
> > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> > "type": "date"
> > },
> > "startedDateTime": {
> > "format": "yyyy-MM-dd HH:mm:ss.SSSSSS",
> > "type": "date"
> > },
> > "signature": {
> > "type": "multi\_field",
> > "fields": {
> > "signature": {
> > "type": "string",
> > "index": "analyzed"
> > },
> > "full": {
> > "type": "string",
> > "index": "not\_analyzed"
> > }
> > }
> > }
> > }
> > }
> > }
> > '
> >
> > On Sun, Jul 10, 2011 at 4:04 PM, Shay Banon
> > \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) ([mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com))\> wrote:
> >
> > > The index template is applied only when the index gets created.
> > >
> > > On Monday, July 11, 2011 at 1:58 AM, Anurag wrote:
> > >
> > > Shay,
> > > Does having a template affect performance? With the bulk index API, is
> > > the template invoked everytime a document is indexed or is there some
> > > sort of internal flag to only invoke template for new indices?
> > >
> > > -anurag
> > >
> > > On Sun, Jul 10, 2011 at 2:28 PM, Shay Banon
> > > \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) ([mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com))\> wrote:
> > >
> > > Check index
> > > templates: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) to
> > > simplify the settings / mappings to automatically apply for this case.
> > >
> > > On Sunday, July 10, 2011 at 10:47 PM, Anurag wrote:
> > >
> > > nm, found it:
> > > "index" : {
> > > "number\_of\_shards":1,
> > > ""number\_of\_replicas":1,
> > > }
> > >
> > > }
> > >
> > > On Sun, Jul 10, 2011 at 12:43 PM, Anurag \<[anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com) ([mailto:anurag.phadke@gmail.com](mailto:anurag.phadke@gmail.com))\> wrote:
> > >
> > > I don't see any latency while its getting indexed, so I guess that's okay.
> > >
> > > On same note, is there a way to specify default # of shards for the
> > > entire cluster?
> > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html)
> > > allows one to specify for a given index, but I don't see a default
> > > systemwide setting as it exists for mapping:
> > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/dynamic-mapping.html)
> > >
> > > -anurag
> > >
> > > On Sun, Jul 10, 2011 at 12:31 PM, Karussell
> > > \<[tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com) ([mailto:tableyourtime@googlemail.com](mailto:tableyourtime@googlemail.com))\> wrote:
> > >
> > > I would use only one shard IMHO (as 5 nodes for 90 indices seems to be
> > > already a high load ;)).
> > >
> > > But check if indexing speed is enough...
> > >
> > > On 10 Jul., 21:27, Anurag \<[anurag.pha...@gmail.com](mailto:anurag.pha...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > >
> > > We are indexing about 3m documents / day (20kb/doc), for a total of
> > > 90-days with 1-replica. Each day corresponds to a new index, thereby
> > > creating 90 indexes. In addition, each index uses a custom mapping,
> > > this mapping is the same for all 90-indexes.
> > >
> > > Given the above scenario, what would be an optimum number of shards
> > > and any other setting that should be considered for a 5-node ES setup?
> > >
> > > -anurag
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:01am UTC](https://discuss.elastic.co/t/optimizing-number-of-shards/4830/11 "2017-07-06T04:01:03Z")