Hello Amit,
Since you have no replicas, increasing the
node_initial_primaries_recoveries is pretty much the only thing you can do
around recovery. There's no real formula about it, since it depends on lots
of variables, like number and size of shards, node hardware, whether or not
nodes will be hit by queries...
What you can do here is to increase that and observe the load and the
startup time/pace when restarting the cluster. When the load gets high (and
you see a decrease of startup times), it means you have too many concurrent
recoveries.
The other area to look at is the design of your indices. You have lots of
them (3000 I assume), so I'd look at ways to get that number way lower.
Because lots of indices which have lots of shards and, in turn, lots of
segments - which means you have a huge amount of files to be read during
recovery. By reducing the number of files you should have better recovery
times due to less disk seeks.
And besides the recovery, shards come with a memory overhead, and the more
shards your searches hit, the slower they get.
If it's not possible to reduce the number of indices, or at least not in
the short term, here are some other things you can do:
For example, with the default "tiered" policy, you can set something
like index.merge.policy.segments_per_tier=4 instead of the default 10. And
to make it work you'll have to set index.merge.policy.max_merge_at_once=4
as well.
Note that tuning the merge policy for less segments will cause more
merging, which will increase the I/O usage during indexing or deleting docs.
Best regards,
Radu
http://sematext.com/ -- Elasticsearch -- Solr -- Lucene
On Wed, Jan 30, 2013 at 3:31 PM, Amit Singh amitsingh.kec@gmail.com wrote:
Hi All,
I have an Elasticsearch with following config;
- 2 nodes- both are data nodes
- Shrads replica- 0
- number of shards -5
The number of active_primary_shards on my cluster is close to 15,000.
When I re-start my cluster it takes 40 to 45 mins to bring up the
cluster. I am running Elasticsearch with default setting.
When I look for the cluster health by;
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
I see that ES is initializing_shards 8 shards at a time. And since I have
15,000 active primary shards.It takes to much time, to bring all the
primary shards into cluster. I looked at the setting to improve this - cluster.routing.allocation.node_initial_primaries_recoveries
to a higher value 25
But I am not sure, should I change this value?
Please help me to understand the different setting that would allow me to
faster cluster re-start. Any formula or logic that would allow me to change
the default setting for faster cluster re-start.
Thanks in advance
Amit
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.