SystemTotalCpuProvider Failed instantiating PerformanceCounter

[marcus.labohm]

Kibana 7.5

Elasticsearch 7.5

APM Server 7.5

APM Agent dot net NuGet Elastic.Apm.NetCoreAll (1.2.0)

Fresh install

.Net Core 2.1 Web App with the Elastic APM NuGet Package.
Deployed on a IIS 8 (8.5.9600) on a Windows Server 2012 R2 Datacenter Version 6.2 (Build 9200)

We see this exception in the logfile of the .net application.

ERROR Elastic.Apm msg:{SystemTotalCpuProvider} Failed instantiating PerformanceCounter - please make sure the current user has permissions to read performance counters. E.g. make sure the current user is member of the 'Performance Monitor Users' group exc:System.UnauthorizedAccessException: Access to the registry key 'Global' is denied. at Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str) at Microsoft.Win32.RegistryKey.InternalGetValueCore(String name, Object defaultValue, Boolean doNotExpand) at System.Diagnostics.PerformanceMonitor.GetData(String item)

I found this post in the Forum wich is pretty simliar.

I used the powershell to performe the TypePerf calls and they return errors. Contrary to the linked post our system uses german language settings and the Performance Counters seems to be configured correctly. I tryed resetting the Performance Counters with the "lodctr /r" command from an elevated PowerShell but this did not seem to change anything.

[GregKalapos]

Thanks for reporting this.

To be honest we have not tested the perf counter part with non-English Windows.

One suspect I have is that on a German Windows it's not possible to query performance counters with their English name. Currently the agent only uses the english perf. counter names.

Also, as a side note, I'm personally not a big fan of perf. counters and I think the best would be to find another way to capture system CPU usage - potentially without perf counters. Mainly because there are countless problems with those.

For this I opened a GitHub issue - that'll hopefully give you a solution: https://github.com/elastic/apm-agent-dotnet/issues/715

Also, just to make sure we are on the same page: all this means is that you won't see system CPU metrics and after the agent tries to query the perf. counter a couple of times it'll stop doing touching the perfcounter - so no system CPU is reported, but everything else should be ok. Is that statement correct in your case?

Beste Grüße
Greg

[marcus.labohm]

Hello Greg and thanks for the fast reply.

Regarding APM we do see Transactions, Errors and, funny enough, CPU usage of the mentioned App. These metrics come from a number of hosts (development, test and production). The Log i mentioned comes from the test system. As you said it stopped after three retrys. We are using Metric Beats to get the overall CPU usage of the test and production systems. The CPU usage of each individual APM is "nice to have" but not required. I did not spend much time with the APM NuGet for .net Core. Is there a way to deactivate the collection of CPU usage?

Kind regards
Marcus

[GregKalapos]

Thanks for the response Marcus,

ok, great to hear - at least the severity of this is then not so high.

Is there a way to deactivate the collection of CPU usage?

In the latest release you can use the MetricsInterval setting and set it to 0. With that you disable all metrics sent by the agent.

I know this is not ideal, so recently a PR got merged and that adds a new setting called DisableMetrics - it is documented here - with that you can add a pattern like this: *cpu* and that’ll disable all metrics that has CPU in their name (list of the metric names are listed in the linked doc) - or just add system.cpu.total.norm.pct to disable this one.

Now, important note: the DisableMetrics setting is not yet in the latest release - the docs I linked above is built from master, but this will be in the next release - I can’t tell you exactly when we release, but we are not far away.

Greg