I recently have been testing the S3 snapshot and restore performance.
I'm able to pull down ~25GB across all 4 data nodes in 10 mins on a
restore. On a given node, I seem to be only getting about 70-80 MBit/s, CPU
utilization is near zero. On the one node that has two shards being
restored to it, the through put is nearly double.
Is there anything I can do to increase the parallelism of the restore
process? So I assume there is something perhaps limited at the shard level?
My Repo Settings
{
"type": "s3",
"settings": {
"bucket": "mybucket",
"region": "us-east",
"protocol": "https",
"base_path": "/elasticsearch",
"secret_key": "SECRET",
"access_key": "KEY",
"max_snapshot_bytes_per_sec": "150mb",
"max_restore_bytes_per_sec": "500mb"
}
}
Cluster Info
ES 1.4.1
4 Data Nodes r3.2xlarge (8 core, 30GB JVM heap, SSD's)
5 Shards, 1 replica
80GB primary store size (160GB w/ replica)
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3cd6348a-9738-4857-9af9-01c3e0b0bafb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.