Improve ElasticSearch Read Cache

Hi there

I'm doing heavy bulk updating on my elasticsearch cluster on a daily base. While is usually works fine on a fresh index (50k updates/sec), after a few bulks the process slows down terribly (1-5k updates/sec). Investigation shows that Elasticsearch keeps hitting the disk for reads (~1'200 K/s reads per process) - I guess it is checking for the existence of the document to be updated.

Is there a way to improve the read cache, so it doesn't has to hit the disk so frequent?

Specs:
24 cores, 36 GB RAM (24GB Heap Size, so 12GB are left for the OS disk cache)

Move to SSDs would be one :slight_smile:

Hi,

Maybe you can make your heap a little smaller and give the OS a bit more RAM and see if that makes a difference?

Otis

Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Elasticsearch Consulting & Support * http://sematext.com/