I'm trying to understand how to scale an Elasticsearch cluster which is extremely slow right now!
Can anyone help?
It's approx. 124,000,000 documents totalling about 60GB.
I index the data once per month. In terms of usage, there are about 10-15 requests per second using a boolean query.
I've done my mappings, only index what needs searching and not use keywords for full-text only fields etc...
Hardware-wise, it's running across two nodes - each 8GB ram & 2 vCPUs. It's on an Elastic Cloud i/o cluster.
So here's the thing.. queries run much quicker on 1 shard (1 pri and 1 rep) than it does on say 3 or 5 shards, even though that 1 shard is 60GB. Can anyone tell me why?
Also I've noticed that CPU usage is through the roof but memory usage is relatively low..
Is the as a result of boolean queries?
Any help would be greatly appreciated!