Heartbeat monitoring not work

Hi All,

I've encountered an issue when finished setup Uptime monitor under Observability. I'm not sure is that my configuration was incorrect or somewhere wrongly config, there're always show 0 monitors

Please see below config file in heartbeat.yml and under heartbeat.monitors.

heartbeat.monitors:

Below are the logs when I run ./heartbeat -e
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.498+0800","log.origin":{"file.name":"instance/beat.go","file.line":707},"message":"Home path: [C:\Program Files\Heartbeat] Config path: [C:\Program Files\Heartbeat] Data path: [C:\Program Files\Heartbeat\data] Logs path: [C:\Program Files\Heartbeat\logs]","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.537+0800","log.origin":{"file.name":"instance/beat.go","file.line":715},"message":"Beat ID: dc1ce55d-4bb2-4555-a2bc-16b93c745cec","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.548+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1081},"message":"Beat info","service.name":"heartbeat","system_info":{"beat":{"path":{"config":"C:\Program Files\Heartbeat","data":"C:\Program Files\Heartbeat\data","home":"C:\Program Files\Heartbeat","logs":"C:\Program Files\Heartbeat\logs"},"type":"heartbeat","uuid":"dc1ce55d-4bb2-4555-a2bc-16b93c745cec"},"ecs.version":"1.6.0"}}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.549+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1090},"message":"Build info","service.name":"heartbeat","system_info":{"build":{"commit":"c2f2aba479653563dbaabefe0f86f5579708ec94","libbeat":"8.4.3","time":"2022-09-27T15:30:49.000Z","version":"8.4.3"},"ecs.version":"1.6.0"}}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.551+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1093},"message":"Go runtime info","service.name":"heartbeat","system_info":{"go":{"os":"windows","arch":"amd64","max_procs":2,"version":"go1.17.12"},"ecs.version":"1.6.0"}}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.556+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1097},"message":"Host info","service.name":"heartbeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2022-10-11T05:14:03+08:00","name":"DESKTOP-UTGRQR7","ip":["fe80::3a:ceb9:5508:b674/64","10.50.10.35/24","::1/128","127.0.0.1/8"],"kernel_version":"10.0.19041.2006 (WinBuild.160101.0800)","mac":["00:0c:29:ce:6b:d7"],"os":{"type":"windows","family":"windows","platform":"windows","name":"Windows 10 Pro","version":"10.0","major":10,"minor":0,"patch":0,"build":"19044.2006"},"timezone":"+08","timezone_offset_sec":28800,"id":"d8eb6bf0-4dd3-4b00-9535-988559c96c03"},"ecs.version":"1.6.0"}}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.557+0800","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1126},"message":"Process info","service.name":"heartbeat","system_info":{"process":{"cwd":"C:\Program Files\Heartbeat","exe":"C:\Program Files\Heartbeat\heartbeat.exe","name":"heartbeat.exe","pid":6304,"ppid":7028,"start_time":"2022-10-11T05:51:43.390+0800"},"ecs.version":"1.6.0"}}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:43.557+0800","log.origin":{"file.name":"instance/beat.go","file.line":293},"message":"Setup Beat: heartbeat; Version: 8.4.3","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"warn","@timestamp":"2022-10-11T05:51:45.917+0800","log.logger":"cfgwarn","log.origin":{"file.name":"tlscommon/config.go","file.line":102},"message":"DEPRECATED: Treating the CommonName field on X.509 certificates as a host name when no Subject Alternative Names are present is going to be removed. Please update your certificates if needed. Will be removed in version: 8.0.0","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:45.918+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: https://NA","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:45.921+0800","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: DESKTOP-UTGRQR7","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:45.921+0800","log.origin":{"file.name":"scheduler/scheduler.go","file.line":79},"message":"limiting to 2 concurrent jobs for 'browser' type","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:45.923+0800","log.origin":{"file.name":"instance/beat.go","file.line":470},"message":"heartbeat start running.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:45.924+0800","log.origin":{"file.name":"beater/heartbeat.go","file.line":100},"message":"heartbeat is running! Hit CTRL-C to stop it.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:45.925+0800","log.origin":{"file.name":"beater/heartbeat.go","file.line":102},"message":"Effective user/group ids: %d/%d, with groups: %v-1 -1 ","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:45.923+0800","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":143},"message":"Starting metrics logging every 30s","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:46.023+0800","log.origin":{"file.name":"logger/logger.go","file.line":97},"message":"Monitor finished","service.name":"heartbeat","event":{"action":"monitor.run"},"monitor":{"id":"auto-http-0X74CD6781A91ED1E8","type":"http","duration":{"ms":92}},"ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.040+0800","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":139},"message":"Connecting to backoff(elasticsearch(https://NA))","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.066+0800","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":162},"message":"'ca_trusted_fingerprint' set, looking for matching fingerprints","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.066+0800","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":173},"message":"CA certificate matching 'ca_trusted_fingerprint' found, adding it to 'certificate_authorities'","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.071+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":291},"message":"Attempting to connect to Elasticsearch version 8.4.1","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.075+0800","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":291},"message":"Attempting to connect to Elasticsearch version 8.4.1","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.077+0800","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":231},"message":"Auto ILM enable success.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.078+0800","log.logger":"index-management.ilm","log.origin":{"file.name":"ilm/std.go","file.line":118},"message":"ILM policy heartbeat exists already.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.090+0800","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":366},"message":"Set settings.index.lifecycle.name in template to {heartbeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_primary_shard_size":"50gb"}}}}}}} as ILM is enabled.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.097+0800","log.logger":"template_loader","log.origin":{"file.name":"template/load.go","file.line":115},"message":"Template "heartbeat-8.4.3" already exists and will not be overwritten.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.097+0800","log.logger":"index-management","log.origin":{"file.name":"idxmgmt/std.go","file.line":267},"message":"Loaded index template.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.100+0800","log.logger":"publisher_pipeline_output","log.origin":{"file.name":"pipeline/client_worker.go","file.line":147},"message":"Connection to backoff(elasticsearch(https://NA)) established","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.186+0800","log.origin":{"file.name":"beater/heartbeat.go","file.line":115},"message":"Ending run_once run","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.188+0800","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":193},"message":"Total metrics","service.name":"heartbeat","monitoring":{"metrics":{"beat":{"cpu":{"system":{"ticks":156,"time":{"ms":156}},"total":{"ticks":374,"time":{"ms":374},"value":374},"user":{"ticks":218,"time":{"ms":218}}},"info":{"ephemeral_id":"6d5334c8-7cee-4bec-b8dd-7d253dd5010e","name":"heartbeat","uptime":{"ms":3758},"version":"8.4.3"},"memstats":{"gc_next":8197824,"memory_alloc":4615392,"memory_sys":18246248,"memory_total":13844464,"rss":44531712},"runtime":{"goroutines":16}},"heartbeat":{"browser":{"endpoint_starts":0,"endpoint_stops":0,"monitor_starts":0,"monitor_stops":0},"http":{"endpoint_starts":1,"endpoint_stops":1,"monitor_starts":1,"monitor_stops":1},"icmp":{"endpoint_starts":0,"endpoint_stops":0,"monitor_starts":0,"monitor_stops":0},"scheduler":{"jobs":{"active":0,"missed_deadline":0},"tasks":{"active":0,"waiting":0}},"tcp":{"endpoint_starts":0,"endpoint_stops":0,"monitor_starts":0,"monitor_stops":0}},"libbeat":{"config":{"module":{"running":0,"starts":0,"stops":0},"reloads":0,"scans":0},"output":{"events":{"acked":1,"active":0,"batches":1,"dropped":0,"duplicates":0,"failed":0,"toomany":0,"total":1},"read":{"bytes":3933,"errors":0},"type":"elasticsearch","write":{"bytes":5693,"errors":0}},"pipeline":{"clients":0,"events":{"active":0,"dropped":0,"failed":0,"filtered":0,"published":1,"retry":1,"total":1},"queue":{"acked":1,"max_events":4096}}},"system":{"cpu":{"cores":2},"handles":{"open":319}}},"ecs.version":"1.6.0"}}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.190+0800","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":194},"message":"Uptime: 3.7616133s","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.191+0800","log.logger":"monitoring","log.origin":{"file.name":"log/log.go","file.line":161},"message":"Stopping metrics logging.","service.name":"heartbeat","ecs.version":"1.6.0"}/
{"log.level":"info","@timestamp":"2022-10-11T05:51:47.192+0800","log.origin":{"file.name":"instance/beat.go","file.line":475},"message":"heartbeat stopped.","service.name":"heartbeat","ecs.version":"1.6.0"}/

Thank you
BR,
CC Ong

Hi All, here'a an update. I realized that the problem from Win10-Services heartbeat is not automatically running. When I start it manually and it will stopped after I refreshed the services page.

Hi @cheching, so, just to confirm, did you manage to solve the problem? Was it that Heartbeat wasn't starting as you expected as you described above?

In the meantime, in case you actually can't see the monitors still, I'd recommend double checking the clock on the machines running Heartbeat, and querying the synthetics* indices to ensure that the summary documents for the monitor runs are there and have the correct times. Then, I'd recommend trying to update the date filters to match those dates to ensure everything is okay.

Please let me know if you have any further help :slight_smile:

Hi @lucasfcosta

Thanks for your reply, I've sort it out for this issue already.

Will let you know if need any help from you. Thanks again

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.