Designing elasitcsearch node spec - CPU cores

Elastic Stack Elasticsearch
1

I not so sure whether this kind of question had been asked before, but I'm having problem on deciding how many cores should I put for my data nodes.

I somehow know how many RAM or disk space I should assign, but for CPU cores, I not entirely sure.

Is the more better? Or even 4 cores should be enough for a typical data nodes?

What are the factors I should look at in determining how many CPU cores needed in data nodes?

Does the amount of CPU cores will affect concurrently how many search request can a data node work on?

Thank you!

2

This depends a lot on the use case but also what type of storage you have and what load you expect the cluster to be under. To get an answer I therefore think you need to provide more details or run some tests.

3

Let's say they are 2 use case here:

  1. Map index, around 5 mil docs, concurrent users expected around 30 users, non-heavy search.

  2. Huge index, perhaps more than 10mil of docs, will be used for data processing hence search request will keep on hitting the nodes, encounter search queue exceed kind of error before.

4

That is not nearly enough information. The size of the data set, type of data and queries have an impact, as does the query and indexing load and latency requirements.

Ideally you want a well-balanced node. There is little point in having lots of CPU if you have slow storage and this limits how fast you can retrieve data to process. If you on the other hand have a small data set that can be cached, you may get limited by CPU even if you have slow storage as disk I/O will be infrequent.

I would recommend running a test to see how much CPU your use case uses or is able to use and make sure that you have at least that amount ton ensure CPU is not a bottleneck.

5

Storage wise, the nodes will be hosted on virtual machine environment, so I think IO will be if-not-same-as-HDD-speed-then-is-slower-than kinda speed.

So does it mean more CPU won't help at all if I/O not good enough?

So is the above the factors you need to look at when determining # of CPU cores?

6

It is hard to predict, so I would always recommend running a test or benchmark.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.