# Metricbeat writes data randomly, instead every 10 sec

**URL:** <https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [September 8, 2020, 9:59am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865 "2020-09-08T09:59:48Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [September 8, 2020, 9:59am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/1 "2020-09-08T09:59:48Z")

</div>

Hi,

We got a problem.  
Metricbeat writes data randomly even though the period=10s in case the query instance is set to all "\*". The data are recorded randomly for specific instance, but not every 10 seconds.  
If I set the query instance to a fixed instance e.g. "\_Total" data comes exactly every 10 sec.  
Tested and noticed problems for metricbeat versions 7.4.x, 7.8.x, 7.9.x.  
However, it works until version 7.3.2.

The servers are not overloaded.  
We have the same problem on all ElasticSearch environments where is metricbeat version higher than 7.3.2.

```
- module: windows
  metricsets:
    - perfmon
  period: 10s
  perfmon.ignore_non_existent_counters: true    
  perfmon.counters:
    - instance_label: logical_disk.name
      measurement_label: logical_disk.Disc.bytes.per_sec
      query: '\LogicalDisk(*)\Disk Bytes/sec' 

```

 ![metric](https://us1.discourse-cdn.com/elastic/original/3X/8/c/8c5a943f2eaa382d4dbdb97a8df253f889f89005.png)

Thanks!

Regards,  
Tadej

---

<div class="post-metadata">

**Author:** ![MarianaD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marianad/32/42687_2.png) [@MarianaD](https://discuss.elastic.co/u/MarianaD)\
**Post date:** [September 9, 2020, 12:55pm UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/2 "2020-09-09T12:55:41Z")

</div>

hi @Razby, it looks like sometimes the event is not returned. Can you set the `logging.level: debug` in the metricbeat.yml file and run metricbeat again.  
Can you check for any exceptions and provide them to us?

---

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [September 10, 2020, 11:19am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/3 "2020-09-10T11:19:09Z")

</div>

Hi,

I set "logging.level: debug" and started the service again. There are no errors or warnings in the log, except those below which are expected, but data still writes randomly instead every 10 sec. ☹

```auto
DEBUG	[perfmon]	perfmon/data.go:47	Ignoring the first measurement because the data isn't ready	{"error": "The data is not valid.", "perfmon": {"query": "\\\\TORX\\LogicalDisk(C:)\\Disk Bytes/sec"}}
WARN	[cfgwarn]	perfmon/config.go:99	DEPRECATED: perfmon.counters configuration option is deprecated and will be removed in the future major version, we advise using the perfmon.queries configuration option instead. Will be removed in version: 8.0

```

 ![metric](https://us1.discourse-cdn.com/elastic/original/3X/4/d/4d0c6420822fe507ea8265b0f1e57458ef697a1b.png)

Regards,  
Tadej

---

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [September 14, 2020, 7:26am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/4 "2020-09-14T07:26:06Z")

</div>

Does really nobody else has a problem with randomly data writing instead predefined interval (period ) e.g. each 10 sec? 🤔

I have this problem on all environments whether it is a newly created environment or an old existing one. 😟

---

<div class="post-metadata">

**Author:** ![MarianaD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marianad/32/42687_2.png) [@MarianaD](https://discuss.elastic.co/u/MarianaD)\
**Post date:** [September 15, 2020, 7:58am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/5 "2020-09-15T07:58:36Z")

</div>

hi @Razby, if there were no errors in this case, can you check also in the logs if the missing events are indeed sent to es or they are not published at all?

---

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [September 15, 2020, 12:22pm UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/6 "2020-09-15T12:22:58Z")

</div>

Hi @MarianaD

I switched metricbeat.exe to the 32-bit version and it works. 🤗  
Apparently there is a bug in the 64-bit version. 🤔  
Do you already know for that bug? Will you be able to check this out?

Regards,  
@razby

---

<div class="post-metadata">

**Author:** ![MarianaD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marianad/32/42687_2.png) [@MarianaD](https://discuss.elastic.co/u/MarianaD)\
**Post date:** [September 15, 2020, 1:27pm UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/7 "2020-09-15T13:27:40Z")

</div>

hi @Razby, we had some issues with the different versions, have you tested with 7.9.1 with the 64 bit version? We recently fixed an issue related to that.  
If you are encountering this on 7.9.1 can you still let us if the missing events are still being published (should be found in the logs)

---

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [September 16, 2020, 7:07am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/8 "2020-09-16T07:07:55Z")

</div>

Hi @MarianaD

I also tried with the version 7.9.1 (64-bit) and it also doesn't work. Events are still missing.  
I checked the logs but missing events were not published. No **publish** event exists in logs for **missing** events. 😖

---

<div class="post-metadata">

**Author:** ![MarianaD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marianad/32/42687_2.png) [@MarianaD](https://discuss.elastic.co/u/MarianaD)\
**Post date:** [September 28, 2020, 1:06pm UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/9 "2020-09-28T13:06:01Z")

</div>

hi @Razby, thanks for testing this out, can you provide us with the os version you are currently running , I have been testing on Windows 10 and Windows Server 2019 and I am not able atm to reproduce the scenario.

---

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [October 1, 2020, 8:04am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/10 "2020-10-01T08:04:05Z")

</div>

Hi @MarianaD  
I have been testing on 64-bit OS Windows 10 and Win Server 2016 DataCenter.  
The problems are perfmon counters with all instances "\*". There is no problem if i use static instance e.g. "\_Total" 🤔

---

<div class="post-metadata">

**Author:** ![MarianaD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marianad/32/42687_2.png) [@MarianaD](https://discuss.elastic.co/u/MarianaD)\
**Post date:** [October 5, 2020, 2:32pm UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/11 "2020-10-05T14:32:27Z")

</div>

hi @Razby, can you set `perfmon.ignore_non_existent_counters` to `false` and again enable debug logging in the `metricbeat.yml` file and provide us with the logs again, we do expect to see errors if the expand path win 32 api has failed.

---

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [October 8, 2020, 6:07am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/12 "2020-10-08T06:07:13Z")

</div>

OK, I will try this.

---

<div class="post-metadata">

**Author:** ![Razby](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/razby/32/127582_2.png) [@Razby](https://discuss.elastic.co/u/Razby)\
**Post date:** [October 8, 2020, 6:51am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/13 "2020-10-08T06:51:04Z")

</div>

Hi @MarianaD  
I set `perfmon.ignore_non_existent_counters` to `false` and enabled debug logging.

Here are the logs that are between the intervals where the events were lost [MetricBeat\_logs](http://razby.si/metricbeatLogs.txt)

 ![metricbeat](https://us1.discourse-cdn.com/elastic/original/3X/0/9/0943701f810fd27620748b0e255dc018611ce1b4.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 5, 2020, 6:51am UTC](https://discuss.elastic.co/t/metricbeat-writes-data-randomly-instead-every-10-sec/247865/14 "2020-11-05T06:51:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.