Elastic cloud one 58 GB node vs AWS ES 4 nodes with 8 GB of memory each performance

Elastic Stack Elasticsearch
1

Hi, I'm planning to use an elastic cloud for my project but I have some confusion about what is the right way to configure my elasticsearch architecture, according to price calculator https://cloud.elastic.co/pricing in order to configure a cluster with 4 data nodes I should buy 4 nodes with 58 GB of memory each but it's way too many resources for my needs which will cost me a lot of money if we take a look at AWS ES there I can buy 4 nodes with 8 GB of memory each, which is perfect for me, with elastic cloud I can't configure such cluster because of a threshold of minimal 58 GB memory.

My question is what solution will be the best in terms of performance and fault tolerance:

  1. Set up 4 nodes with 8 GB of memory each on AWS ES and if one node falls down another 3 will make a job (P.S. I understand limitations of AWS ES and they don't bother me)
  2. Set up 1 node with 58 GB of memory on elastic cloud but here I don't know what will happen if this node falls down? will my data be transferred to another 58 GB node automatically, which can cause downtime, or I will be waiting for current node recovery?
2

Welcome!

Why do you think you need 4 nodes?

3

dadoonet, We have about 40 GB of data and to split a load and achieve fault tolerance I decided that 4 nodes will be enoughth

4

Why not starting smaller and see how much HEAP/nodes you really need?

I mean that IMHO, if the data indexed on disk is 40gb, you can most likely use a single primary shard or 2 to hold the data. Then, 2 shards could fit in few gb of HEAP. I'd start with a cloud instance of 4gb, test and if it does not work, increase to 8gb (4gb of HEAP)...

Then, I'd probably use 2 zones. So something like:

image
image1992×818 70.6 KB

Not sure if you need more than that. "It depends" of course. As usual...
What is your use case typically?

1 Like
5

I think you are right here. One question I would like to clarify is if I choose to set up an elastic cloud on AWS will I be charged for data transfer between aws rds, s3, dynamodb and elastic cloud? If my AWS resources located in the same region as an elastic cloud then I shouldn't but I may be wrong.

6

I honestly don't know. May be create a free trial account on cloud.elastic.co and ask the support? Unless @uricohen knows?

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.