Understanding cpu metrics

[asp]

Hi all,

I am running ELK 5.1.2 as single instance on RHEL7.x. I also installed basic license to activate monitoring.
We made some loadtest of a new application which should be monitored in ELK, which contained a new logstash parser and many new log lines.

I wanted to check if our ELK environment is fast enough for that.

Our machine (running ELK + Apache for proxying kibana with TLS) configured in the following way:
CPU of system: 8
ES: processors: 6
LS: pipeline.workers: 4

We are also running metricbeat once a minute for monitoring cpu, mem of system and important processes.

metricbeat (cpu sum, user, system):

-> we see max CPU at 51%

When I check the process CPU load, I see following:

-> elasticsearch 508% at max
-> logstash 209% at max.

Checking ES in Monitoring, I see the following:

-> 100% CPU of ES at max.

Now the thing I do not understand:

• Montoring says 6 defined processors are used by 100%.
• I expected to see 600% in beats per process.
• If I have 500 % (ES) and 200% LS in beats per process, then I expect 700% in sum of CPU ( plus a bit by other processes).
• but why is the cpu usage of system (user + system) only at 50%?

in beats I used following fields for my analysis:
for process cpu: system.process.cpu.total.pct
for system wide cpu: system.cpu.user.pct and system.cpu.system.pct. the total cpu is the sum of both by scripted field.

Thanks, Andreas

[tudor (Tudor Golubenco)]

Just a guess, perhaps you have hyper-threading so the max is 1600%?

[asp]

at least not as I understand the output of lscpu:

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    1
Core(s) per socket:    2
Socket(s):             4
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Model name:            Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
Stepping:              2
CPU MHz:               2399.684
BogoMIPS:              4799.99
Hypervisor vendor:     VMware
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              20480K
NUMA node0 CPU(s):     0-7