Cluster configuration advice

Elastic Stack Elasticsearch
1

Hello,

I need to store logs generated by say 50 apps and the average log size a day is 5 GB with a retention period of a month, the write speed isn't important for me but the read time is, high availability isn't also that essantial.

so what do you guys think about the right cluster configuration for this case?

Thanks in advance <3

2

Hello,

Total Data for one month retention. = 5 * 31 = 155 GB
Total Storage (GB) = 155+ (155 * 5% Margin of error ) = 163 GB
Total Data Nodes = ROUNDUP( 163 / Memory per data node / Memory:data ratio) + 1 Data node for failover capacity.

So a 3-node cluster to provide a reliable solution. 3 master data nodes with a 1 shard / index.

1 Like
3

@ahmed_charafouddine i didn't get the ROUNDUP part, can you explain more?

4

ROUNDUP is just a mathematical function for rounding up decimal numbers.

How much storage per VM can you have?

5

the storage might be 250gb per node but not sure, i think the number of cores per cpu also matters and i don't think i can have more than core i5

6

You're absolutely right. In this case, having 3 nodes with 4 cores,> = 16GB of RAM and 1 shard / day / India could be enough in my opinion

7

Also, the clarity / quality of your data, the mapping / the setting analysis could also affect the performance of your cluster

8

it's good to know, thank you for the clear response!

9

@ahmed_charafouddine may i ask why 16gb of RAM?

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.