CPU Core binding on Elastissearch

Elastic Stack Elasticsearch
1

We have single node server with 256 GB RAM and 32 core processor. We are running elasticsearch , logstash and kibana.

We want to bind the no of cores (example 8) for elasticsearch process only. Is there any configuration in .yml file?

Awaiting for your valuable response.

Thanks,

2

I think it's called processors and you can take a look at the link below
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-threadpool.html#processors

3

That's not CPU core binding. That setting controls how Elasticsearch will size all it's internal thread pools, which is not CPU core binding. This is something only OS kernel can do.

4

I could be wrong but if you review the page, you'll see the settings with regard to the relationship between the number of threads and the number of processors for each thread pool in a node.

To me, it provides info related to the question in a sense if they want to create a dedicated node for a specific type of operation. Also, it provides inside info about how ES determines the number of threads for each pool based on the number of processors (or cores) available within a system.

Lastly, this paragraph at the processors setting is very important to know when running multiple instances of ES in one physical machine

  1. If you are running multiple instances of Elasticsearch on the same host but want Elasticsearch to size its thread pools as if it only has a fraction of the CPU, you should override the processors setting to the desired fraction (e.g., if you’re running two instances of Elasticsearch on a 16-core machine, set processors to 8). Note that this is an expert-level use-case and there’s a lot more involved than just setting the processors setting as there are other considerations like changing the number of garbage collector threads, pinning processes to cores, etc.
5

This is all correct, yet, this is not CPU core binding in the sense that threads are assigned to the be executed on specific CPU cores. This can only be done using OS kernel features such as containers, for instance.

2 Likes
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.