I'm doing heavy bulk updating on my elasticsearch cluster on a daily base. While is usually works fine on a fresh index (50k updates/sec), after a few bulks the process slows down terribly (1-5k updates/sec). Investigation shows that Elasticsearch keeps hitting the disk for reads (~1'200 K/s reads per process) - I guess it is checking for the existence of the document to be updated.
Is there a way to improve the read cache, so it doesn't has to hit the disk so frequent?
24 cores, 36 GB RAM (24GB Heap Size, so 12GB are left for the OS disk cache)