Virtual bytes wrong counter in metricbeat


#1

I use perfmon to check Virtual Bytes for a process currently but we are trying to metricbeat to push the information elsewhere. The Virtual Bytes in metricbeat never matches even close to what is shown by perfmon or PS get-counter for \Virtual Bytes, I think it might be the wrong counter.

Metric - what the doc says - what I found
system.process.memory.size - The total virtual memory the process has. - Appears to actually be the Private Bytes

system.process.memory.rss.bytes - The Resident Set Size. The amount of memory the process occupied in main memory (RAM). -Appears to actually be the Working Set

system.process.memory.share - The shared memory the process uses. - Shows as 0 for me so I'm not sure.

Anyone else experiance this?

Windows 2008 / MetricBeat 6.4.0 x64
module config:
metricbeat.modules:

  • module: system
    metricsets:
    • process
      enabled: true
      period: 60s
      processes: ['w3wp.*']


#2

I also tried using the windows\perfmon metricsets. Unfortunately that doesn't give me the metric with command line so I can't search for the specific process I need. There are multiple w3wp processes and I need to know the app pool somehow to get the one I need.


(Jaime Soriano) #3

Hi @noskills and welcome :slight_smile:

As you mention, system.process.memory.size are actually the Private Bytes, and system.process.memory.rss.bytes the Working Set Size.
In the implementation of process memory collection on Windows, only these two values are used, you can find it here: https://github.com/elastic/gosigar/blob/v0.9.0/sigar_windows.go#L318

Memory usage reporting is tricky, specially here in metricbeat, that is intended to have a consistent set of metrics even in different platforms.
memory.size should be a good approximation of the memory resources the process is using by itself,
do you think Virtual Bytes would be a better approximation than Private Bytes for this value?
memory.share is currently not being calculated in Windows, do you think it could be calculated in function of the other values?


#4

@jsoriano,
From my understanding the virtual bytes is useful because it includes all the different locations the memory could be in use. For a 32-bit application you are limited to 4GB. So the private bytes don't include all of the memory used by an application (dlls), or shared so if the process hits the 4GB limit you would have issues.
In my case we are graphing it so we can ensure not too many objects are loaded into memory and hitting the cap.
Just clarified with someone on 32-bit apps. One of things virtual bytes gives you is what the application has asked for and never released. So if the process runs for a long time it can still hit the limit and have issues. You can see a huge gap in my screenshot. The size reported by metricbeat is 700MB while Virtual bytes is already at 1589MB, more than double the reported.

Thanks,
Justin


(Jaime Soriano) #5

Hi @noskills,

I have created a new github issue to follow with this. Thanks!


#6

@jsoriano
Awesome, I'll keep an eye on it! Thanks for the input and listening.


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.