I have a perfectly running cluster on AWS (not managed), that is composed of 6x M5D instances.
So we have local ssd storage (we got rid of EBS since this has caused troubles), 2 CPU cores and some RAM.
For costs reason, each node is master eligible, data node & client node and we have a loadbalancer in front of them.
The total cost is about 587$ for the instances.
Our cluster contains now 90 indices that have 6 shards and 2 replicas each, and about 190M documents.
We are adding every two month a new instance because we tends to run out of storage and costs are going up.
So I'm wondering if we are doing good or not with this setup, maybe it's time for a redesign?
Cluster needs to meet those points:
- "100% uptime" (should be "as reliable as possible" )
- Can loss 2 nodes at anytime without disruption/losing data
- Cheap to run & effective
Current search rate is about 200/s and indexing rate is between 900 and 1500/s.
Oh, and we do bulk indexing and could have some complex queries.