You have decided to put 200 shards on a single node. Such a high number
combined with a significant shard size can not be recovered very quick.
Although you can put many thousands shards on a single node and shards can
grow into many hundreds of GB, you always have a price to pay when the
shards start moving around between nodes at recovery time.
The improvement depends on how much you want to stress a node while
recovering. The default settings are chosen wisely so when a node comes up,
it can always respond to search and index requests immediately while
recovery. It does not confuse clients with timeouts, and it does not
confuse sysadmins with iowaits. But there is a long down time, and this
down time is directly configured by the number of shards per node and the
current shard size.
So if you want to stress your nodes, you can select a higher value in
- recovery takes many network resources
- queries and indexing may not respond in time
- higher iowaits
- network bandwidth must be available
The best method to achieve quick recovery is to select a wise shards per
node ratio and a sane shard size.
Here is my approach: on my 32-core machines, I plan to never run more than
32 shards, and no shard shall grow beyond 5-10g, so transporting on a
10GBit/s takes reasonable time.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firstname.lastname@example.org.
For more options, visit https://groups.google.com/groups/opt_out.