create indexes with 30 shards one by one,when shards number up to 10000 in elasticsearch cluster,i found creation operation goes very slow.
Why do you want to create so many shards? Do you have more than 100 data nodes?
30 data nodes,we have more than 600 index.
Which version of Elasticsearch are you using? Do you have dedicated master nodes? What is the average shard size?
yes ,we have dedicated master nodes. average shard size is small,we just created index, the data is not in yet,it looks like there is no matter with shard size. we used Elasticsearch 2.0.
If the shards are small, why do you need a primary shard per node? Do you know how much data the indices will hold?
One index will hold 600G,so we set 30 shards to a index.
If the index holds 600GB of data, 30 primary shards might be a reasonable starting point as that is 20GB per shard. Will all indices hold the same amount of data? If not, I would recommend setting up smaller indices with fewer primary shards.
What is the use case?
Actually,it will. Do you have some idea?
I am not sure I am following what you are referring to.
What is the specification with respect to CPU and RAM for your dedicated master nodes?
Upgrade. Many things changed since then.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.