Max Number of data nodes per machines

[avnere (Avner)]

Hi,

I want to install few data nodes on one machines. Each data node will get 32GB RAM.
Is there any formula for getting the max number of nodes per machine CPU and RAM?

Thanks

[rugenl (Len Rugen)]

I think the official word is "this is not recommended", but I'd say somewhere between total_ram/32 -1 and total_ram/64. So a 128G machine could hold 2-3 nodes.

Assuming your 32G is for heap, Elastic has moved a lot of memory usage off-heap, so if these are production, be careful.

[warkolm (Mark Walkom)]

Elasticsearch can be pretty intensive on resources so contention needs to be taken into account. If the host can handle it then go for it.

[avnere (Avner)]

What about the CPU?

Basically, I am running benchmark, I am indexing ~300GB.
I have machine with 512 RAM and 72 cores.

I placed on it 1 master node + 4 data node. The result is ~400k index rate.
When changing the cluster to 2 machine with 2 data nodes on each I am getting ~600k index rate.

[rugenl (Len Rugen)]

My guess is that unless you have exceptional io system, cpu won't be the bottleneck. You are benching your environment, that is always the best answer.