Improper detection of available CPUs

Elastic Stack Elasticsearch
1

Im running a 5 node cluster in kubernetes. Each k8 host has 56 cores but when I query for node information I get these values:

    "available_processors" : 1,
    "allocated_processors" : 1

Im using Java 12 so according to this the correct value is supposed to be passed to the container(s).

Furthermore - if I open a shell to a running container and run nproc I get the correct value (56).

Have I misconfigured something? Do I need to manually specify # of cores to the JVM?

  • Similar issue to this posting
2

Hi @ethrbunny

It is necessary to set a limit in your Kubernetes manifest. For example, if my host had 4 cores, it would be:

containers:
- name: elasticsearch
  image: docker.elastic.co/elasticsearch/elasticsearch:7.3.0
  resources:
    requests:
      cpu: 500m 
    limits:
      memory: 4

Regards :vulcan_salute:

Rodrigo

1 Like
3

So don't change the JVM arguments?

NM - I think I get it now. Will have a go at it.

TY

4

Hi, @ethrbunny

I have not changed any JVM arguments. I have set a limit cpu and it worked :slight_smile:. There are some blogs on Internet that talk about this kind of issue. It depends on Java version and the proper application.

Regards :vulcan_salute:

1 Like
5

(sorry to keep bringing this up - ran into some confusing info)

Ran into this doc entry:

If you do not specify a CPU limit for a Container, then one of these situations applies:

* The Container has no upper bound on the CPU resources it can use. The Container could use all of the CPU resources available on the Node where it is running.

I checked the container definition(s) for my data nodes and there isn't a CPU request or limit set.

Does the cpu request / limit have to be set no matter what?

6

Hi, @ethrbunny

If you don't set a resource & limit on Kubernetes, that pod will use all resources of the machine.

However, in this case, Elasticsearch is a Java application. So that, it needs a CPU limit in order to take it account.

:warning: Not set a memory limit, Elasticsearch use file cache.

Regards :vulcan_salute:,

Rodrigo

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.