# Memory metrics not correct

**URL:** <https://discuss.elastic.co/t/memory-metrics-not-correct/261688>\
**Category:** Metrics\
**Created:** [January 20, 2021, 6:29pm UTC](https://discuss.elastic.co/t/memory-metrics-not-correct/261688 "2021-01-20T18:29:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jodinathan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jodinathan/32/45821_2.png) [@jodinathan](https://discuss.elastic.co/u/jodinathan)\
**Post date:** [January 20, 2021, 6:29pm UTC](https://discuss.elastic.co/t/memory-metrics-not-correct/261688/1 "2021-01-20T18:29:10Z")

</div>

Hi,

I am using Metrics with docker but the numbers seem to be incorrect.

Docker stats:

 ![Screen Shot 2021-01-20 at 09.27.26](https://us1.discourse-cdn.com/elastic/original/3X/2/c/2c3cd411993697cf8222726fa30bafe8a2edfad3.png)

Memory in Metrics:  
 ![Screen Shot 2021-01-20 at 09.28.43](https://us1.discourse-cdn.com/elastic/original/3X/f/c/fca47d2f1ab9d4874382cf8137b6b4f4683decc1.png)

My docker-compose:

```auto
  app:
    ...
  monitor:
    container_name: monitor
    build:
      context: ../metricbeat
    environment:
      - appName=$appName
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /proc:/hostfs/proc:ro
      - /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro
      - /:/hostfs:ro

```

metricbeat Dockerfile

```auto
FROM docker.elastic.co/beats/metricbeat:7.10.2

COPY ./metricbeat.yml /usr/share/metricbeat/metricbeat.yml

```

metricbeat.yml

```auto
name: "${appName}"
tags: ["dart-app"]

metricbeat.config.modules:
  path: /usr/share/metricbeat/modules.d/*.yml
  reload.enabled: true

processors:
  - add_cloud_metadata:

output.elasticsearch:
  hosts: ["xxx"]

setup.kibana:
  host: "xxx"

```

---

<div class="post-metadata">

**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\
**Post date:** [January 21, 2021, 10:47am UTC](https://discuss.elastic.co/t/memory-metrics-not-correct/261688/2 "2021-01-21T10:47:57Z")

</div>

Hi @jodinathan,

it's possible that the docker cli and the Metrics UI look at different time intervals and calculate the averages differently. Can you compare the change of the values over time from both tools to see if that shows any correlation? Does the node details page show the same deviating values?

---

<div class="post-metadata">

**Author:** ![jodinathan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jodinathan/32/45821_2.png) [@jodinathan](https://discuss.elastic.co/u/jodinathan)\
**Post date:** [January 25, 2021, 1:06pm UTC](https://discuss.elastic.co/t/memory-metrics-not-correct/261688/3 "2021-01-25T13:06:43Z")

</div>

Hi @weltenwort,

I am not using node, it is a Dart application. By looking at the Observatory page, the memory usage is close to the Docker stats.  
I analyzed a range metric and it is the same behavior.

Digging the Kibana dashboard, it seems that the percentage is used + cached memory and that may be the reason for the discrepancy.

what do you think?

---

<div class="post-metadata">

**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\
**Post date:** [January 25, 2021, 3:22pm UTC](https://discuss.elastic.co/t/memory-metrics-not-correct/261688/4 "2021-01-25T15:22:07Z")

</div>

> I am not using node, it is a Dart application.

Sorry for using ambiguous terms here. With "node" I meant the box on the inventory screen. It might be a host, a container or a pod depending on the selected view.

> it seems that the percentage is used + cached memory

Your hypothesis sounds plausible. The metricbeat module collects the stats via the docker API for which I found the following note in the `docker stats` docs:

> On Linux, the Docker CLI reports memory usage by subtracting cache usage from the total memory usage. The API does not perform such a calculation but rather provides the total memory usage and the amount from the cache so that clients can use the data as needed. The cache usage is defined as the value of `total_inactive_file` field in the `memory.stat` file on cgroup v1 hosts.
> 
> On Docker 19.03 and older, the cache usage was defined as the value of `cache` field. On cgroup v2 hosts, the cache usage is defined as the value of `inactive_file` field.

([source](https://docs.docker.com/engine/reference/commandline/stats/#extended-description))

So `docker stats` performs some custom post-processing of the data to arrive at these values, which likely don't match that of many monitoring tools.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 22, 2021, 3:22pm UTC](https://discuss.elastic.co/t/memory-metrics-not-correct/261688/5 "2021-02-22T15:22:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.