New community beat Perfmonbeat

Hi @all,

i would like to show you a new community beat i created. It's called Perfmonbeat and it is a beat to collect perfomance counters from windows. It's yet very basic and there is a lot more to do ( exception handling, configs, naming of outputs, ...). But i want to discuss with you if it goes in the right direction and if there is a need for such a beat.

See here



1 Like

I definitely like the idea of collecting perf data through the PDH API.

It looks like this would work nicely as a MetricSet in Metricbeat. Instead of using the beat generator there is a metricset generator that creates a standalone beat using Metricbeat as a framework. Then you would have a MetricSet that could be later integrated into Metricbeat or be used as a plugin.

I didn't review the code since it's a work in progress. But one thing you should consider doing is generating code for the methods you need in the PDH API (here's a place where we do that in gosigar). It uses the mksyscall_windows tool from Go.

And if you require constants from header files you can do some generation there too. You would create a defs_windows.go file that contains cgo code then run a generator to build another go file. See defs_windows.go.

This makes the code a little cleaner and means that you don't need to use cgo for your builds which then allows for easier cross-compiles.

1 Like

Hi @andrewkroh, many thanks for that detailed answer!! Especially for that part with cgo!! Really exciting stuff :open_mouth: I also think it is better to create this as a MetricSet. I just wanted to start with coding. I create a new metricset an come back if i'm done.

Hi @andrewkroh, i have create a new repo to build a MetricSet instead of a beat. Now i'm try to configure config settings for that metricset. Assume the following structure i created

        - perfmon

windows is my module and perfmon my metricset. Where i can add config only for that metricset

- module: windows
  metricsets: ["perfmon"]
  enabled: true
  period: 1s
  hosts: ["localhost"]
    - alias: "Prozessorzeit"
      query: "\\Prozessorinformationen(*)\\Prozessorzeit (%)"

I have found it in the documentation

Hi @andrewkroh, i have created file defs_windows.go with the following content

package perfmon

#include <windows.h>
#include <stdio.h>
#include <conio.h>
#include <pdh.h>
#include <pdhmsg.h>
#cgo LDFLAGS: -lpdh
import "C"

const (
	PDH_NO_DATA             = C.PDH_NO_DATA
	PdhFmtDouble            = C.PDH_FMT_DOUBLE
	PdhFmtLarge             = C.PDH_FMT_LARGE
	PdhFmtLong              = C.PDH_FMT_LONG


i then call go tool cgo -godefs defs_windows.go > defs_windows_amd64.go which creates this output

// Created by cgo -godefs - DO NOT EDIT
// cgo.exe -godefs defs_windows.go

package perfmon

const (
	PDH_NO_DATA		= 0x800007d5
	PDH_STATUS_NO_OBJECT	= 0xc0000bb8
	PDH_INVALID_HANDLE	= 0xc0000bbc
	PDH_INVALID_DATA	= 0xc0000bc6
	PDH_NO_MORE_DATA	= 0xc0000bcc
	PdhFmtDouble		= 0x200
	PdhFmtLarge		= 0x400
	PdhFmtLong		= 0x100

type PdhCounterValue struct {
	CStatus		uint32
	Pad_cgo_0	[4]byte
	LongValue	int32
	Pad_cgo_1	[4]byte

But if i then call go build or go generate i get this error \perfmon\defs_windows_amd64.go: unexpected NUL in input. Any ideas??

Do you have a // +build ignore at the top of the defs_windows.go file?

Should that command be present in all def files? defs_windows_amd64.go, defs_windows_386.go. Is it necessary to create these 386, amd64 files or is it better to copy the output from the cgo command into defs_windows.go and just have this file? There is also one thing more :see_no_evil: i created a wrapper pdh_windows.go like you say.

//go:generate go run $GOROOT/src/syscall/mksyscall_windows.go -output pdh_windows.go pdh.go
// Windows API calls
//sys   _PdhOpenQuery(dataSource *string, userData int, query *syscall.Handle) (err int) = pdh.PdhOpenQuery
//sys   _PdhAddCounter(query syscall.Handle, counterPath string, userData int, counter *syscall.Handle) (err int) = pdh.PdhAddCounter
//sys   _PdhCollectQueryData(query syscall.Handle) (err int) = pdh.PdhCollectQueryData
//sys   _PdhGetFormattedCounterValue(counter syscall.Handle, format int, counterType int, value PdhCounterValue) (err int) = pdh.GetFormattedCounterValue

This all works fine, except in the output file there is an import statement "". which is used by modpdh = windows.NewLazySystemDLL("pdh.dll"). should i use syscall instead and then modpdh = syscall.NewLazyDLL("pdh.dll")??

Sorry for all that questions :disappointed_relieved:

defs_windows.go is the source file for generating defs_windows_amd64.go and defs_windows_386.go. All three files should be committed to your repo. defs_windows.go will not be used by go build so it needs to have the build tag so that the compiler ignore it. Doing this allows us to build without needing cgo. cgo is only needed if we have to change defs_windows.go and need to regenerate defs_windows_amd64.go and defs_windows_386.go.

What "output" file? The code generated by mksyscall_windows.go should have imports for syscall and not windows.

pdh_windows.go. This is the generated file from mksyscall_windows.go. There is an import for windows. Maybe a bug or and old version??

It looks like it does have logic for the package. I didn't examine the logic, but I would prefer functions in the syscall package over if you have a choice (just to reduce the external deps). But if there's a function in the windows package that makes life easier use it.

Ok, i will use the syscall package. Here are the new metricset i have created (ignore the name of the beat). There is only that error with unexpteced NUL in defs_windows_amd64.go. Maybe you have some time to look over. I can't see whats wrong with this file :confused:

Where's the error coming from?

It builds for me:

$ pwd
$ GOOS=windows GOARCH=amd64 go build; echo $?

That error comes if i'm trying to build on windows.

Ohh, got it. This error throws if you are in the wrong folder. I tried go build in maddin2016/pdhbeat instead of maddin2016\pdhbeat\module\windows. How can i test if the new metricset is working? Do i have to build the whole metricbeat?

go build in will produce pdhbeat.exe that you can use. If you uncomment this line you can use the the normal metricbeat modules too (the configuration will all be namespaced under pdhbeat instead of metricbeat).

Check defs_windows.go for bad characters. Run gofmt on it and gofmt on the output files too. I'm seeing bad encoding.

defs_windows_amd64.go:1:1: illegal UTF-8 encoding
defs_windows_amd64.go:1:1: illegal character U+FFFD '�'
defs_windows_amd64.go:1:2: illegal UTF-8 encoding

It seems that the go tool cgo command add these characters. I had to explicit save the file as UTF-8 with notepad.

Hi @andrewkroh, if you had the time can try to run this metricbeat on a windows machine. I had to do some fixing of types and add some errors. But basically it is running.

@maddin2016 with go make sure your files are UTF-8 always. By default windows tools often use UTF16LE which might not parse well with utf-8.

Yeap, comming from .NET UTF16 is the default if you not specify encoding.