Startup Type missing data

Elastic Stack Beats
,
1

Hey all,

We have been looking into the Windows Service metricset, and have found that none of the services are coming through with Startup Type data. We are following the metricset configuration - and there doesn't even seem to be a way to filter out that field if we wanted to.

We validated the information is not making it into logstash - metricbeat is not producing the data. Has anyone else come across this?

"@metadata": {
 "beat": "metricbeat",
 "type": "_doc",
 "version": "7.8.0"
},

"service": {
 "type": "windows"
},

"windows": {
	"service": {
	 "uptime": {
		"ms": 519639883
	},
 "id": "2Ok94HqlSi",
	"display_name": "World Wide Web Publishing Service",
 "pid": 4876,
 "start_type": "",
	"path_name": "C:\\Windows\\system32\\svchost.exe -k iissvcs",
	"start_name": "localSystem",
 "name": "W3SVC",
 "state": "Running"
 }
},

"ecs": {
	"version": "1.5.0"
},

"host": {
 "name": "seisp13pltrpt01",
	"id": "9749c806-4325-4edd-b920-a7ba99fa14c4",
	"ip": [
		"10.50.40.120",
	 "fe80::5efe:a32:2878"
 ],
	"mac": [
	 "00:50:56:b6:42:df",
		"00:00:00:00:00:00:00:e0"
 ],
	"hostname": "seisp13pltrpt01",
 "architecture": "x86_64",
	"os": {
		"name": "Windows Server 2012 R2 Standard",
		"kernel": "6.3.9600.19846 (winblue_ltsb_escrow.200923-1735)",
		"build": "9600.19873",
	 "platform": "windows",
	 "version": "6.3",
		"family": "windows"
 }
},

"agent": {
	"name": "seisp13pltrpt01",
	"type": "metricbeat",
	"version": "7.8.0",
	"hostname": "seisp13pltrpt01",
 "ephemeral_id": "e2024c3f-1f1a-4af7-b4b6-f816234916b3",
	"id": "6980b6cb-8abd-446a-9d96-a7fefef87159"
},

"event": {
	"dataset": "windows.service",
 "module": "windows",
	"duration": 76001700
},

"metricset": {
 "period": 60000,
	"name": "service"
}

}
2

Hi @mjsteckiel :slightly_smiling_face:

Can you paste your Metricbeat configuration and some logs from your running metricbeat with metricbeat -e -d "*", please?

Just to confirm, your windows is Windows Server 2012 R2 Standard, right?

3

Hi @Mario_Castro

What @mjsteckiel has posted above is from the debug log of one of the metricbeats. We have a few different versions running but that from 7.8.0 metricbeat. Looking at line 19 we can see that "start_type" is showing blank. So it appears the local beat is not able to, or is not trying, to pick up the value.
Yes, the servers in question are 2012 R2 and 2016 standard servers

Thoughts?
thanks

4

Hi @mgevans :slightly_smiling_face:

I have checked and there are no known issues with your Win version so everything should be fine about that.

About the debug logs, this is how they look like:

./metricbeat -e                                                                                                                                                                                                                              [22:06:40]
2020-12-09T22:06:43.618+0100	INFO	instance/beat.go:647	Home path: [/home/mcastro/go/src/github.com/elastic/beats/metricbeat] Config path: [/home/mcastro/go/src/github.com/elastic/beats/metricbeat] Data path: [/home/mcastro/go/src/github.com/elastic/beats/metricbeat/data] Logs path: [/home/mcastro/go/src/github.com/elastic/beats/metricbeat/logs]
2020-12-09T22:06:43.619+0100	INFO	instance/beat.go:655	Beat ID: 7a0a00bb-faa3-4b2b-9d92-4a0a17bcb8f3
2020-12-09T22:06:43.634+0100	INFO	[seccomp]	seccomp/seccomp.go:124	Syscall filter successfully installed
2020-12-09T22:06:43.634+0100	INFO	[beat]	instance/beat.go:976	Beat info	{"system_info": {"beat": {"path": {"config": "/home/mcastro/go/src/github.com/elastic/beats/metricbeat", "data": "/home/mcastro/go/src/github.com/elastic/beats/metricbeat/data", "home": "/home/mcastro/go/src/github.com/elastic/beats/metricbeat", "logs": "/home/mcastro/go/src/github.com/elastic/beats/metricbeat/logs"}, "type": "metricbeat", "uuid": "7a0a00bb-faa3-4b2b-9d92-4a0a17bcb8f3"}}}
2020-12-09T22:06:43.634+0100	INFO	[beat]	instance/beat.go:985	Build info	{"system_info": {"build": {"commit": "unknown", "libbeat": "8.0.0", "time": "1754-08-30T22:43:41.128Z", "version": "8.0.0"}}}
2020-12-09T22:06:43.634+0100	INFO	[beat]	instance/beat.go:988	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":12,"version":"go1.14.4"}}}
2020-12-09T22:06:43.636+0100	INFO	[beat]	instance/beat.go:992	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-12-09T19:54:25+01:00","containerized":false,"name":"anonymous","ip":["127.0.0.1/8","::1/128","192.168.1.20/24","fe80::2521:9883:12a5:7947/64","192.168.16.1/20","172.27.0.1/16","172.22.0.1/16","172.26.0.1/16","fe80::42:33ff:feff:fe8c/64","172.17.0.1/16","172.23.0.1/16","172.30.0.1/16","192.168.80.1/20","172.31.0.1/16","192.168.224.1/20","172.24.0.1/16","192.168.128.1/20","192.168.32.1/20","192.168.160.1/20","fe80::42:74ff:fef6:800e/64","172.21.0.1/16","192.168.96.1/20","192.168.112.1/20","192.168.48.1/20","172.20.0.1/16","192.168.208.1/20","172.25.0.1/16","192.168.192.1/20","172.18.0.1/16","172.29.0.1/16","192.168.64.1/20","172.28.0.1/16","192.168.144.1/20","fe80::b042:a7ff:fe7f:dcf7/64","fe80::ec88:c3ff:fe6d:f80b/64","fe80::d08d:85ff:feac:e2de/64"],"kernel_version":"5.6.10-arch1-1","mac":["9c:b6:d0:b9:7e:d3","02:42:83:68:b6:49","02:42:3b:65:5c:22","02:42:a9:85:26:46","02:42:33:ff:fe:8c","02:42:ee:b4:06:82","02:42:c9:07:f9:92","02:42:40:4b:df:08","02:42:f7:38:26:a9","02:42:46:82:63:da","02:42:49:33:f3:0a","02:42:e6:09:5f:0b","02:42:36:5f:85:e4","02:42:a7:2f:4c:64","02:42:74:f6:80:0e","02:42:9b:7b:7d:ec","02:42:e8:bb:8b:12","02:42:b4:94:ae:4c","02:42:6b:53:38:53","02:42:52:e1:68:f8","02:42:8d:e0:71:a5","02:42:ee:e3:49:9e","02:42:88:35:d9:b6","02:42:6d:f1:c8:8c","02:42:62:c0:30:24","02:42:39:e8:92:c9","02:42:46:37:b5:92","02:42:75:4a:4d:3d","b2:42:a7:7f:dc:f7","ee:88:c3:6d:f8:0b","d2:8d:85:ac:e2:de"],"os":{"family":"","platform":"antergos","name":"Antergos Linux","version":"","major":0,"minor":0,"patch":0},"timezone":"CET","timezone_offset_sec":3600,"id":"54f70115bae545cbac2b150f254472a0"}}}
2020-12-09T22:06:43.637+0100	INFO	[beat]	instance/beat.go:1021	Process info	{"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":null,"effective":null,"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/home/mcastro/go/src/github.com/elastic/beats/metricbeat", "exe": "/home/mcastro/go/src/github.com/elastic/beats/metricbeat/metricbeat", "name": "metricbeat", "pid": 288960, "ppid": 6297, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2020-12-09T22:06:43.060+0100"}}}
2020-12-09T22:06:43.637+0100	INFO	instance/beat.go:302	Setup Beat: metricbeat; Version: 8.0.0
2020-12-09T22:06:43.637+0100	INFO	[index-management]	idxmgmt/std.go:184	Set output.elasticsearch.index to 'metricbeat-8.0.0' as ILM is enabled.
2020-12-09T22:06:43.637+0100	INFO	eslegclient/connection.go:99	elasticsearch url: http://localhost:9200
2020-12-09T22:06:43.637+0100	INFO	[publisher]	pipeline/module.go:113	Beat name: anonymous
2020-12-09T22:06:43.640+0100	INFO	instance/beat.go:466	metricbeat start running.
2020-12-09T22:06:43.640+0100	INFO	[monitoring]	log/log.go:118	Starting metrics logging every 30s
2020-12-09T22:06:43.640+0100	INFO	filesystem/filesystem.go:57	Ignoring filesystem types: sysfs, tmpfs, bdev, proc, cgroup, cgroup2, cpuset, devtmpfs, binfmt_misc, configfs, debugfs, tracefs, securityfs, sockfs, bpf, pipefs, ramfs, hugetlbfs, devpts, autofs, efivarfs, mqueue, pstore, fuse, fusectl, overlay
2020-12-09T22:06:43.640+0100	INFO	[system.fsstat]	fsstat/fsstat.go:57	Ignoring filesystem types: %ssysfs, tmpfs, bdev, proc, cgroup, cgroup2, cpuset, devtmpfs, binfmt_misc, configfs, debugfs, tracefs, securityfs, sockfs, bpf, pipefs, ramfs, hugetlbfs, devpts, autofs, efivarfs, mqueue, pstore, fuse, fusectl, overlay
2020-12-09T22:06:43.640+0100	INFO	cfgfile/reload.go:164	Config reloader started
2020-12-09T22:06:43.642+0100	INFO	filesystem/filesystem.go:57	Ignoring filesystem types: sysfs, tmpfs, bdev, proc, cgroup, cgroup2, cpuset, devtmpfs, binfmt_misc, configfs, debugfs, tracefs, securityfs, sockfs, bpf, pipefs, ramfs, hugetlbfs, devpts, autofs, efivarfs, mqueue, pstore, fuse, fusectl, overlay
2020-12-09T22:06:43.642+0100	INFO	[system.fsstat]	fsstat/fsstat.go:57	Ignoring filesystem types: %ssysfs, tmpfs, bdev, proc, cgroup, cgroup2, cpuset, devtmpfs, binfmt_misc, configfs, debugfs, tracefs, securityfs, sockfs, bpf, pipefs, ramfs, hugetlbfs, devpts, autofs, efivarfs, mqueue, pstore, fuse, fusectl, overlay
2020-12-09T22:06:43.643+0100	INFO	cfgfile/reload.go:224	Loading of config files completed.
^C2020-12-09T22:06:44.374+0100	INFO	cfgfile/reload.go:227	Dynamic config reloader stopped
2020-12-09T22:06:44.374+0100	INFO	[reload]	cfgfile/list.go:129	Stopping 3 runners ...
2020-12-09T22:06:46.625+0100	INFO	[add_cloud_metadata]	add_cloud_metadata/add_cloud_metadata.go:101	add_cloud_metadata: hosting provider type not detected.
2020-12-09T22:06:46.627+0100	INFO	[monitoring]	log/log.go:153	Total non-zero metrics	{"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"cfs":{"period":{"us":100000}},"id":"user.slice"},"cpuacct":{"id":"user.slice","total":{"ns":9562150479985}},"memory":{"id":"user@1000.service","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":15487926272}}}},"cpu":{"system":{"ticks":120,"time":{"ms":125}},"total":{"ticks":350,"time":{"ms":356},"value":350},"user":{"ticks":230,"time":{"ms":231}}},"handles":{"limit":{"hard":524288,"soft":1024},"open":9},"info":{"ephemeral_id":"2c5f4feb-9351-4d57-b6e9-275f43075b24","uptime":{"ms":3049}},"memstats":{"gc_next":11050144,"memory_alloc":8130960,"memory_total":63122320,"rss":63807488},"runtime":{"goroutines":20}},"libbeat":{"config":{"module":{"running":3,"starts":3},"reloads":1,"scans":1},"output":{"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":10,"failed":5,"published":10,"total":15}}},"metricbeat":{"system":{"cpu":{"events":1,"success":1},"filesystem":{"events":2,"success":3},"fsstat":{"events":1,"success":1},"load":{"events":1,"success":1},"memory":{"events":1,"success":1},"network":{"events":4,"success":5},"process":{"events":2,"success":3},"process_summary":{"events":1,"success":1},"socket_summary":{"events":1,"success":1},"uptime":{"events":1,"success":1}}},"system":{"cpu":{"cores":12},"load":{"1":1.77,"15":2.39,"5":2.24,"norm":{"1":0.1475,"15":0.1992,"5":0.1867}}}}}}
2020-12-09T22:06:46.627+0100	INFO	[monitoring]	log/log.go:154	Uptime: 3.050257558s
2020-12-09T22:06:46.627+0100	INFO	[monitoring]	log/log.go:131	Stopping metrics logging.
2020-12-09T22:06:46.627+0100	INFO	instance/beat.go:472	metricbeat stopped.

Sometimes you can get useful hints about why a field is missing in an event.

The configuration yaml is also important because sometimes it's just some indentation error there.

5

@Mario_Castro Thanks. Yes, we know a debug file :smiley: I copied the relevant part to spare the world from many lines of text. I've messaged you the entire contents.

here are our two relevant yml files. Pretty much straight out of the box. We are also running the IIS module and System module on this particular box but keep in mind we have no startup_type data from any of our couple dozen systems in our test environment.
modules.d\windows.yml

# Module: windows
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.8/metricbeat-module-windows.html
- module: windows
  metricsets:
    - service
  period: 1m

and metricbeat.yml (comment lines removed)

metricbeat.config.modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
    
setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
  
setup.kibana:  

output.logstash:
    hosts: ["11.22.33.44:5044"]
            
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
6

Hi Mario! I am new to elastic (and monitoring), so Michael stepped in to give you the info you asked for (thanks, Michael!), but I just wanted to say hi and thank you for looking at this!

7

Hi All,
Just a quick note that the new 7.10.1 agents are sending data and we can view it in Kibana.
Our 7.8.0 agents are still not sending the Startup_Type data.
If you're using 7.8.0 or 7.9.1 agents I suspect you'll have the same bug.
We'll let this close

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.