# Metricbeat Missing Data

**URL:** <https://discuss.elastic.co/t/metricbeat-missing-data/320285>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [December 1, 2022, 7:08pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285 "2022-12-01T19:08:46Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tim\_Mobley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_mobley/32/94741_2.png) [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\
**Post date:** [December 1, 2022, 7:08pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285/1 "2022-12-01T19:08:46Z")

</div>

I have Metricbeat receiving data from Perfmon counters, including Process \> `% Processor Time`. this is configured to collect every 10 seconds. However, we are missing processes for some reason. In other words, the events are not showing all the processes that we know are running on the machine at a given time. When we look in Perfmon directly, we can see the missing processes, but not in Kibana. Is there a reason Metricbeat is missing these data? At first we thought it was due to the 10s setting, but the process is running constantly and should be picked up at one of these intervals. It is crucial that we not miss any information - for troubleshooting reasons.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 1, 2022, 7:16pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285/2 "2022-12-01T19:16:01Z")

</div>

Hi @Tim_Mobley

> [@Tim\_Mobley](#):
>
> Is there a reason Metricbeat is missing these data?

Please share

`modules.d/system.yml`

and look at this in detail

There are many options... the default collects the Top 5, there are comfigurations to collect more, specific etc..etc..

> **[System process metricset | Metricbeat Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-process.html)**

> **`process.include_top_n`**  
> These options allow you to filter out all processes that are not in the top N by CPU or memory, in order to reduce the number of documents created. If both the `by_cpu` and `by_memory` options are used, the union of the two sets is included.

and the default is the following...

```auto
  process.include_top_n:
    by_cpu: 5 # include top 5 processes by CPU
    by_memory: 5 # include top 5 processes by memory

```

So you will probably want to adjust / take out etc..

If you take it out you pretty sure you will get every process and it will be quite verbose.

Also you can use filtering

**`processes`**

> When the `process` metricset is enabled, you can use the `processes` option to define a list of regexp expressions to filter the processes that are reported. For more complex filtering, you should use the `processors` configuration option. See [_Processors_](https://www.elastic.co/guide/en/beats/metricbeat/current/filtering-and-enhancing-data.html) for more information.

---

<div class="post-metadata">

**Author:** ![Tim\_Mobley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_mobley/32/94741_2.png) [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\
**Post date:** [December 1, 2022, 7:52pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285/3 "2022-12-01T19:52:06Z")

</div>

Here's my `system.yml`:

```auto
- module: system
  period: 10s
  metricsets:
    - cpu
    - core
    - memory
    - network
    - process
    - process_summary
    - socket_summary
  process.include_cpu_ticks: true
  process.include_top_n:
    by_cpu: 10
    by_memory: 10

- module: system
  period: 1m
  metricsets: 
    - filesystem
    - fsstat
  processors: 
    - drop_event.when.regexp:
        system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib|snap)(%|/)'

- module: system
  period: 15m
  metricsets:
    - uptime

```

So, if I wanted to see all the processes not matter what their utilization, should I just omit the `process.include_top_n` section?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 1, 2022, 7:52pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285/4 "2022-12-01T19:52:26Z")

</div>

Yup!

BTW I see no diskio.. 😉

---

<div class="post-metadata">

**Author:** ![Tim\_Mobley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_mobley/32/94741_2.png) [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\
**Post date:** [December 1, 2022, 8:17pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285/5 "2022-12-01T20:17:25Z")

</div>

Do you recommend that monitoring `diskio` metric as a standard practice?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 1, 2022, 9:49pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285/6 "2022-12-01T21:49:37Z")

</div>

> [@Tim\_Mobley](#):
>
> Do you recommend that monitoring `diskio` metric as a standard practice?

Depends on what is important to you... Some of the default dashboards have diskio metrics so those viz will be empty if diskio is not on...

But if diskio is not important (say for stateless apps) then no need

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 29, 2022, 11:50pm UTC](https://discuss.elastic.co/t/metricbeat-missing-data/320285/7 "2022-12-29T23:50:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.