# \>30s to create an index
**URL:** https://discuss.elastic.co/t/30s-to-create-an-index/10693
**Category:** Elasticsearch
**Created:** [February 11, 2013, 1:46pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693 "2013-02-11T13:46:03Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Filirom1](https://discuss.elastic.co/u/Filirom1)
#### Post date: [February 11, 2013, 1:46pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/1 "2013-02-11T13:46:03Z")
Hi guys,
On a 4 nodes cluster a new index creation takes more than 30 seconds.
Is it something usual ?
$ time curl -XPUT [http://localhost:9023/test/user/kimchy](http://localhost:9023/test/user/kimchy) -d '{
"name" : "Shay Banon"
}'
{"ok":true,"\_index":"test","\_type":"user","\_id":"kimchy","\_version":1}
real 0m32.836s
user 0m0.002s
sys 0m0.005s
I enabled the cluster.service logs to see what happens. I put the logs on a
gist [https://gist.github.com/Filirom1/4754342](https://gist.github.com/Filirom1/4754342)
Cheers
Romain
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@drewr](https://discuss.elastic.co/u/drewr)
#### Post date: [February 11, 2013, 4:08pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/2 "2013-02-11T16:08:17Z")
Filirom1 wrote:
> On a 4 nodes cluster a new index creation takes more than 30 seconds.
> Is it something usual ?
>
> $ time curl -XPUT [http://localhost:9023/test/user/kimchy](http://localhost:9023/test/user/kimchy) -d '{
> "name" : "Shay Banon"
> }'
> {"ok":true,"\_index":"test","\_type":"user","\_id":"kimchy","\_version":1}
> real 0m32.836s
> user 0m0.002s
> sys 0m0.005s
>
> I enabled the cluster.service logs to see what happens. I put the logs on a
> gist [ElasticSearch log with cluster.service in DEBUG · GitHub](https://gist.github.com/Filirom1/4754342)
It looks like you've created, or are creating, a bunch of unique
indices and types as you index. Can we see the indexing code?
Perhaps you're doing something inefficiently we can help you think
through.
-Drew
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Filirom1](https://discuss.elastic.co/u/Filirom1)
#### Post date: [February 11, 2013, 4:39pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/3 "2013-02-11T16:39:22Z")
When I did the curl command to create a new index, the cluster was doing
nothing else.
The indexing code is the curl command.
This 4 nodes cluster is not used in production. It just for test purpose,
so I can try what you want on it.
2013/2/11 Drew Raines [aaraines@gmail.com](mailto:aaraines@gmail.com)
> Filirom1 wrote:
>
> > On a 4 nodes cluster a new index creation takes more than 30 seconds.
> > Is it something usual ?
> >
> > $ time curl -XPUT [http://localhost:9023/test/user/kimchy](http://localhost:9023/test/user/kimchy) -d '{
> > "name" : "Shay Banon"
> > }'
> > {"ok":true,"\_index":"test","\_type":"user","\_id":"kimchy","\_version":1}
> > real 0m32.836s
> > user 0m0.002s
> > sys 0m0.005s
> >
> > I enabled the cluster.service logs to see what happens. I put the logs
> > on a
> > gist [ElasticSearch log with cluster.service in DEBUG · GitHub](https://gist.github.com/Filirom1/4754342)
>
> It looks like you've created, or are creating, a bunch of unique
> indices and types as you index. Can we see the indexing code?
> Perhaps you're doing something inefficiently we can help you think
> through.
>
> -Drew
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@egaumer](https://discuss.elastic.co/u/egaumer)
#### Post date: [February 11, 2013, 6:57pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/4 "2013-02-11T18:57:17Z")
Are there any other indices in the cluster? Types?
It looks like you have a lot of mappings that are continuously getting
refreshed.
On Monday, February 11, 2013 8:46:03 AM UTC-5, Filirom1 wrote:
> Hi guys,
>
> On a 4 nodes cluster a new index creation takes more than 30 seconds.
> Is it something usual ?
>
> $ time curl -XPUT [http://localhost:9023/test/user/kimchy](http://localhost:9023/test/user/kimchy) -d '{
> "name" : "Shay Banon"
> }'
> {"ok":true,"\_index":"test","\_type":"user","\_id":"kimchy","\_version":1}
> real 0m32.836s
> user 0m0.002s
> sys 0m0.005s
>
> I enabled the cluster.service logs to see what happens. I put the logs on
> a gist [ElasticSearch log with cluster.service in DEBUG · GitHub](https://gist.github.com/Filirom1/4754342)
>
> Cheers
> Romain
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Filirom1](https://discuss.elastic.co/u/Filirom1)
#### Post date: [February 12, 2013, 10:17am UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/5 "2013-02-12T10:17:43Z")
There are 51 indices in the cluster, and for each indices there is 6 shards
and 1 replica.
Here is the number of types per index :
[ 6,
9,
6,
1,
15,
2,
3,
31,
1,
3,
4,
2,
0,
3,
5,
5,
86,
4,
1,
50,
1,
0,
1,
3,
21,
3,
8,
51,
6,
3,
2,
27,
2,
318,
3,
711,
2,
4,
21,
11,
3,
2,
2,
4,
5,
3,
1,
3,
8,
9,
1 ]
The download the \_mapping, it weights 5.2Mo
2013/2/11 egaumer [egaumer@gmail.com](mailto:egaumer@gmail.com)
> Are there any other indices in the cluster? Types?
>
> It looks like you have a lot of mappings that are continuously getting
> refreshed.
>
> On Monday, February 11, 2013 8:46:03 AM UTC-5, Filirom1 wrote:
>
> > Hi guys,
> >
> > On a 4 nodes cluster a new index creation takes more than 30 seconds.
> > Is it something usual ?
> >
> > $ time curl -XPUT [http://localhost:9023/test/\*\*user/kimchy](http://localhost:9023/test/**user/kimchy)[http://localhost:9023/test/user/kimchy](http://localhost:9023/test/user/kimchy)-d '{
> > "name" : "Shay Banon"
> > }'
> > {"ok":true,"_index":"test","_ **type":"user","_id":"kimchy","_**
> > version":1}
> > real 0m32.836s
> > user 0m0.002s
> > sys 0m0.005s
> >
> > I enabled the cluster.service logs to see what happens. I put the logs on
> > a gist [https://gist.github.com/\*\*Filirom1/4754342](https://gist.github.com/**Filirom1/4754342)[https://gist.github.com/Filirom1/4754342](https://gist.github.com/Filirom1/4754342)
> >
> > Cheers
> > Romain
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [February 12, 2013, 12:54pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/6 "2013-02-12T12:54:51Z")
As said, you created many indices, each with many types, with many
shards, and even more dynamic fields that are refreshing dynamically.
Remember: each new field mapping is a cluster upgrade. Such an operation
is costly as it propagates throughout the whole cluster.
30 seconds are very fast according to the some thousands of cluster
state field mapping upgrade operations you have provoked.
The index creation time scales over the number of nodes, indices, types,
shards, and fields. Use less and you will observe faster creation times.
Best regards,
Jörg
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Filirom1](https://discuss.elastic.co/u/Filirom1)
#### Post date: [February 12, 2013, 1:02pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/7 "2013-02-12T13:02:38Z")
Thanks a lot for the explanation !
2013/2/12 Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com)
> As said, you created many indices, each with many types, with many shards,
> and even more dynamic fields that are refreshing dynamically.
>
> Remember: each new field mapping is a cluster upgrade. Such an operation
> is costly as it propagates throughout the whole cluster.
>
> 30 seconds are very fast according to the some thousands of cluster state
> field mapping upgrade operations you have provoked.
>
> The index creation time scales over the number of nodes, indices, types,
> shards, and fields. Use less and you will observe faster creation times.
>
> Best regards,
>
> Jörg
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscribe@\*\*[googlegroups.com](http://googlegroups.com)[elasticsearch%2Bunsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com)
> .
> For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out)
> .
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [February 12, 2013, 10:54pm UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/8 "2013-02-12T22:54:22Z")
I would still love to see the cluster state if you provide, would like to simulate it on our end and see if we can improve things. Can you gist the result of /\_cluster/state on the cluster with all the indices?
On Feb 12, 2013, at 2:02 PM, Romain [filirom1@gmail.com](mailto:filirom1@gmail.com) wrote:
> Thanks a lot for the explanation !
>
> 2013/2/12 Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com)
> As said, you created many indices, each with many types, with many shards, and even more dynamic fields that are refreshing dynamically.
>
> Remember: each new field mapping is a cluster upgrade. Such an operation is costly as it propagates throughout the whole cluster.
>
> 30 seconds are very fast according to the some thousands of cluster state field mapping upgrade operations you have provoked.
>
> The index creation time scales over the number of nodes, indices, types, shards, and fields. Use less and you will observe faster creation times.
>
> Best regards,
>
> Jörg
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Filirom1](https://discuss.elastic.co/u/Filirom1)
#### Post date: [February 13, 2013, 8:38am UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/9 "2013-02-13T08:38:50Z")
I would prefer not to publish the mapping on internet. I sent it to you by
email.
Thank you
Romain
2013/2/12 [kimchy@gmail.com](mailto:kimchy@gmail.com)
> I would still love to see the cluster state if you provide, would like to
> simulate it on our end and see if we can improve things. Can you gist the
> result of /\_cluster/state on the cluster with all the indices?
>
> On Feb 12, 2013, at 2:02 PM, Romain [filirom1@gmail.com](mailto:filirom1@gmail.com) wrote:
>
> Thanks a lot for the explanation !
>
> 2013/2/12 Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com)
>
> > As said, you created many indices, each with many types, with many
> > shards, and even more dynamic fields that are refreshing dynamically.
> >
> > Remember: each new field mapping is a cluster upgrade. Such an operation
> > is costly as it propagates throughout the whole cluster.
> >
> > 30 seconds are very fast according to the some thousands of cluster state
> > field mapping upgrade operations you have provoked.
> >
> > The index creation time scales over the number of nodes, indices, types,
> > shards, and fields. Use less and you will observe faster creation times.
> >
> > Best regards,
> >
> > Jörg
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to elasticsearch+unsubscribe@\*\*[googlegroups.com](http://googlegroups.com)[elasticsearch%2Bunsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com)
> > .
> > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out)
> > .
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:51am UTC](https://discuss.elastic.co/t/30s-to-create-an-index/10693/10 "2017-07-06T02:51:44Z")