# Uptime heartbeat service is failing **URL:** \ **Category:** Elastic Observability\ **Tags:** elastic-stack-monitoring\ **Created:** [January 23, 2025, 8:19am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558 "2025-01-23T08:19:54Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![Amol\_Nagotkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amol_nagotkar/32/124188_2.png) [@Amol\_Nagotkar](https://discuss.elastic.co/u/Amol_Nagotkar)\ **Post date:** [January 23, 2025, 8:19am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/1 "2025-01-23T08:19:54Z")
Hi all, I am trying observability uptime monitoring. elastic .17.1 kibana .17.1 heartbeat 7.17.1 ```auto ############################# Heartbeat ###################################### # Directory to load monitor definitions heartbeat.config.monitors: path: ${path.config}/monitors.d/*.yml reload.enabled: true reload.period: 5s # Configure monitors inline heartbeat.monitors: - type: http enabled: true id: s1 name: s1 urls: ["http://local_ip:9010/health-check"] schedule: '@every 10s' timeout: 30s # Added timeout to avoid indefinite hangs - type: http enabled: true id: s2 name: s2 urls: ["http://local_ip:9201/health-check"] schedule: '@every 10s' timeout: 30s # Added timeout to avoid indefinite hangs # ======================= Elasticsearch template setting ======================= setup.template.settings: index.number_of_shards: 1 index.codec: best_compression # =================================== Kibana =================================== setup.kibana: host: "http://local_ip:5601" # Explicitly added "http://" # ================================== Outputs =================================== output.elasticsearch: hosts: ["http://local_ip:9200"] # Explicitly added "http://" # ================================= Processors ================================= processors: - add_observer_metadata: geo: name: us-east-1a location: "37.926868, -78.024902" # Example geo-location # ================================== Logging =================================== logging.level: info # Default is 'info'; change to 'debug' if needed # ============================= X-Pack Monitoring ============================== monitoring.enabled: true # Set to true if you want to enable Stack Monitoring ``` error -\> ```auto × heartbeat-elastic.service - Ping remote services for availability and log results to Elasticsearch or send to Logstash. Loaded: loaded (/lib/systemd/system/heartbeat-elastic.service; disabled; vendor preset: enabled) Active: failed (Result: core-dump) since Thu 2025-01-23 13:34:01 IST; 1min 18s ago Docs: https://www.elastic.co/beats/heartbeat Process: 1389106 ExecStart=/usr/share/heartbeat/bin/heartbeat --environment systemd $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS (code=dumped, signal=ABRT) Main PID: 1389106 (code=dumped, signal=ABRT) CPU: 82ms Jan 23 13:34:01 VCPL-LP-93 systemd[1]: heartbeat-elastic.service: Scheduled restart job, restart counter is at 29. Jan 23 13:34:01 VCPL-LP-93 systemd[1]: Stopped Ping remote services for availability and log results to Elasticsearch or send to Logstash.. Jan 23 13:34:01 VCPL-LP-93 systemd[1]: heartbeat-elastic.service: Start request repeated too quickly. Jan 23 13:34:01 VCPL-LP-93 systemd[1]: heartbeat-elastic.service: Failed with result 'core-dump'. Jan 23 13:34:01 VCPL-LP-93 systemd[1]: Failed to start Ping remote services for availability and log results to Elasticsearch or send to Logstash.. ``` 1. service is gets fail after few minutes 2. status is updating but circular graph is not updating. plus it takes at least 1 min to update. how can i do it quickly. please see image below. ![Screenshot from 2025-01-23 13-46-21](https://us1.discourse-cdn.com/elastic/original/3X/1/9/192053e099fdec542940a0c3634c6b1508529d07.png) after doing so many times refresh also, its not working. Thank you in advance ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [January 23, 2025, 8:44am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/2 "2025-01-23T08:44:48Z")
Can you provide details from the hearbeat log? ---
**Author:** ![Amol\_Nagotkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amol_nagotkar/32/124188_2.png) [@Amol\_Nagotkar](https://discuss.elastic.co/u/Amol_Nagotkar)\ **Post date:** [January 23, 2025, 8:45am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/3 "2025-01-23T08:45:47Z")
Hi Thanks for quick reply. can you tell me how to get that ? ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [January 23, 2025, 8:50am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/4 "2025-01-23T08:50:39Z")
Should be in /var/log/heartbeat, if is not set, add logging as it's recommended [here](https://www.elastic.co/guide/en/beats/heartbeat/current/configuration-logging.html). ---
**Author:** ![Amol\_Nagotkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amol_nagotkar/32/124188_2.png) [@Amol\_Nagotkar](https://discuss.elastic.co/u/Amol_Nagotkar)\ **Post date:** [January 23, 2025, 9:36am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/5 "2025-01-23T09:36:38Z")
```auto 2025-01-23T13:15:50.315+0530 INFO instance/beat.go:686 Home path: [/usr/share/heartbeat] Config path: [/etc/heartbeat] Data path: [/var/lib/heartbeat] Logs path: [/var/log/heartbeat] Hostfs Path: [/] 2025-01-23T13:15:50.317+0530 INFO instance/beat.go:694 Beat ID: fc3e5b8e-dad6-40df-b42c-d3049c89fe1b 2025-01-23T13:15:50.321+0530 INFO [beat] instance/beat.go:1040 Beat info {"system_info": {"beat": {"path": {"config": "/etc/heartbeat", "data": "/var/lib/heartbeat", "home": "/usr/share/heartbeat", "logs": "/var/log/heartbeat"}, "type": "heartbeat", "uuid": "fc3e5b8e-dad6-40df-b42c-d3049c89fe1b"}}} 2025-01-23T13:15:50.321+0530 INFO [beat] instance/beat.go:1049 Build info {"system_info": {"build": {"commit": "1d05ba86138cfc9a5ae5c0acc64a57b8d81678ff", "libbeat": "7.17.1", "time": "2022-02-23T23:44:00.000Z", "version": "7.17.1"}}} 2025-01-23T13:15:50.321+0530 INFO [beat] instance/beat.go:1052 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":16,"version":"go1.17.6"}}} 2025-01-23T13:15:50.323+0530 INFO [beat] instance/beat.go:1056 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2025-01-22T22:16:15+05:30","containerized":false,"name":"VCPL-LP-93","ip":["127.0.0.1/8","::1/128","10.40.13.27/23","fe80::8368:a1b1:33cb:63aa/64","172.22.0.1/16","172.24.0.1/16","172.27.0.1/16","172.26.0.1/16","172.28.0.1/16","172.19.0.1/16","fe80::42:5cff:fe23:26f/64","172.17.0.1/16","172.25.0.1/16","172.23.0.1/16","fe80::42:59ff:fec2:ef84/64","172.20.0.1/16","fe80::42:2ff:feb8:9b24/64","172.21.0.1/16","fe80::42:22ff:fe2a:93ec/64","172.18.0.1/16","fe80::42:27ff:fe3d:b21d/64","fe80::3887:fcff:fe2b:134d/64","fe80::f0f7:adff:fe12:dbed/64","fe80::cf0:e1ff:fe33:423d/64","fe80::90aa:3cff:feaf:a3f3/64","fe80::5494:a3ff:fe94:825b/64","fe80::7867:f2ff:fe9b:b19d/64","fe80::e803:2ff:fe28:fd93/64","fe80::74cd:90ff:fe46:5936/64","fe80::9c51:79ff:fea8:a01f/64","172.29.0.1/16","fe80::42:8aff:fe32:e295/64","fe80::bc0a:3cff:febf:cab6/64"],"kernel_version":"6.8.0-50-generic","mac":["64:6e:e0:65:10:ee","02:42:77:06:33:5c","02:42:5d:8d:0e:2a","02:42:79:a9:2d:7d","02:42:dc:e8:89:1d","02:42:35:75:83:4f","02:42:5c:23:02:6f","02:42:90:08:a0:96","02:42:ef:07:8c:bb","02:42:59:c2:ef:84","02:42:02:b8:9b:24","02:42:22:2a:93:ec","02:42:27:3d:b2:1d","3a:87:fc:2b:13:4d","f2:f7:ad:12:db:ed","0e:f0:e1:33:42:3d","92:aa:3c:af:a3:f3","56:94:a3:94:82:5b","7a:67:f2:9b:b1:9d","ea:03:02:28:fd:93","76:cd:90:46:59:36","9e:51:79:a8:a0:1f","02:42:8a:32:e2:95","be:0a:3c:bf:ca:b6"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"22.04.5 LTS (Jammy Jellyfish)","major":22,"minor":4,"patch":5,"codename":"jammy"},"timezone":"IST","timezone_offset_sec":19800,"id":"4368346214834eef8ca074b5ede63651"}}} 2025-01-23T13:15:50.324+0530 INFO [beat] instance/beat.go:1085 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["net_raw"],"effective":["net_raw"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"ambient":null}, "cwd": "/etc/heartbeat", "exe": "/usr/share/heartbeat/bin/heartbeat", "name": "heartbeat", "pid": 1237921, "ppid": 1237920, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2025-01-23T13:15:49.270+0530"}}} 2025-01-23T13:15:50.324+0530 INFO instance/beat.go:328 Setup Beat: heartbeat; Version: 7.17.1 2025-01-23T13:15:50.324+0530 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'heartbeat-7.17.1' as ILM is enabled. 2025-01-23T13:15:50.324+0530 INFO [esclientleg] eslegclient/connection.go:105 elasticsearch url: http://10.40.13.27:9200 2025-01-23T13:15:50.324+0530 INFO [publisher] pipeline/module.go:113 Beat name: VCPL-LP-93 ``` ![Screenshot from 2025-01-23 15-12-37](https://us1.discourse-cdn.com/elastic/original/3X/5/2/5251c1e2fb372096db20a040232a42cac5360940.png) ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [January 23, 2025, 11:37am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/6 "2025-01-23T11:37:39Z")
There is no details there. Check for more details in the journal log `journalctl -u heartbeat.service -n --since "2 hour ago"` Also might help to set `logging.level: debug` in heartbeat.yml ---
**Author:** ![Amol\_Nagotkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amol_nagotkar/32/124188_2.png) [@Amol\_Nagotkar](https://discuss.elastic.co/u/Amol_Nagotkar)\ **Post date:** [January 23, 2025, 11:42am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/7 "2025-01-23T11:42:53Z")
![Screenshot from 2025-01-23 17-08-58](https://us1.discourse-cdn.com/elastic/original/3X/b/3/b33f672a4e9eec23550d20c001de238af050eeb0.png) Fatal glibc error: rseq registration failed i am trying different solutions from google, but not able to resolve. it would be great if you give me steps to how to install glibc. ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [January 23, 2025, 7:16pm UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/8 "2025-01-23T19:16:29Z")
> [@Amol\_Nagotkar](#): > > Fatal glibc error: rseq registration failed A similar [problem](https://discuss.elastic.co/t/filebeat-and-glibc-errors-on-ubuntu-22-04/306653). ---
**Author:** ![Amol\_Nagotkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amol_nagotkar/32/124188_2.png) [@Amol\_Nagotkar](https://discuss.elastic.co/u/Amol_Nagotkar)\ **Post date:** [January 24, 2025, 5:56am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/9 "2025-01-24T05:56:44Z")
able to resolve above issue by adding code below in heartbeat.yml file ```auto seccomp: default_action: allow syscalls: - action: allow names: - rseq ``` now please help me to understand .. where is the setting to reduce refresh interval of that circular graph. it takes too much time to refresh/update. ![Screenshot from 2025-01-23 13-46-21](https://us1.discourse-cdn.com/elastic/original/3X/1/9/192053e099fdec542940a0c3634c6b1508529d07.png) and status section takes around 1 min to refresh/update. Thank you in advance. ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [January 24, 2025, 7:25am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/10 "2025-01-24T07:25:43Z")
Not sure why is so slow. If you think about how often will the monitor collect data, there is parameter: `schedule: '@every 10s'` ---
**Author:** ![Amol\_Nagotkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amol_nagotkar/32/124188_2.png) [@Amol\_Nagotkar](https://discuss.elastic.co/u/Amol_Nagotkar)\ **Post date:** [January 24, 2025, 7:27am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/11 "2025-01-24T07:27:25Z")
that i have already done. you can see that in my 1st message. but where is the setting to reduce refresh interval ? ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [January 24, 2025, 7:40am UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/12 "2025-01-24T07:40:40Z")
If you are talking about the dashboard internal, then in Kibana. ![Kibana](https://us1.discourse-cdn.com/elastic/original/3X/d/b/dbd50fee0f599e74888edb2b13728529fc09f7d8.jpeg) ---
**Author:** ![Amol\_Nagotkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amol_nagotkar/32/124188_2.png) [@Amol\_Nagotkar](https://discuss.elastic.co/u/Amol_Nagotkar)\ **Post date:** [January 24, 2025, 12:37pm UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/13 "2025-01-24T12:37:07Z")
i"ll check that. can u tell me how to add ssl configuration in heartbeat.yml file ?? heartbeat.yml file ```auto ############################# Heartbeat ###################################### # Directory to load monitor definitions heartbeat.config.monitors: path: ${path.config}/monitors.d/*.yml reload.enabled: true reload.period: 5s # Configure monitors inline heartbeat.monitors: - type: http enabled: true id: S1 name: S1 urls: ["http://local_ip:9010/health-check"] schedule: '@every 10s' timeout: 30s # Added timeout to avoid indefinite hangs # timeout to avoid indefinite hangs # ======================= Elasticsearch template setting ======================= setup.template.settings: index.number_of_shards: 1 index.codec: best_compression # =================================== Kibana =================================== setup.kibana: host: "https://local_ip:5601" # Explicitly added "http://" ssl: enabled: true # Set to true if using SSL certificate_authorities: ["path_to_cert"] certificate: "path_to_cert" key: "path_to_cert" verification_mode: "certificate" # ================================== Outputs =================================== output.elasticsearch: hosts: ["https://local_ip:9200"] # Explicitly added "http://" ssl: enabled: true # Set to true if using SSL certificate_authorities: ["path_to_cert"] certificate: "path_to_cert" key: "path_to_cert" verification_mode: "certificate" # ================================= Processors ================================= # processors: # - add_observer_metadata: # geo: # name: us-east-1a # location: "37.926868, -78.024902" # Example geo-location # ================================== Logging =================================== logging.level: info # Default is 'info'; change to 'debug' if needed # ============================= X-Pack Monitoring ============================== monitoring.enabled: true # Set to true if you want to enable Stack Monitoring seccomp: default_action: allow syscalls: - action: allow names: - rseq ``` even if adding correct certificate i am getting below error ```auto Jan 24 18:24:59 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:24:59.480+0530 ERROR [esclientleg] transport/logging.go:37 Error dialing x509: certificate signed by unknown author> Jan 24 18:25:13 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:13.397+0530 INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": > Jan 24 18:25:14 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:14.199+0530 ERROR [esclientleg] transport/logging.go:37 Error dialing x509: certificate signed by unknown author> Jan 24 18:25:43 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:43.389+0530 INFO [monitoring] log/log.go:184 Non-zero metrics in the last 30s {"monitoring": {"metrics": > Jan 24 18:25:44 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:44.212+0530 ERROR [esclientleg] transport/logging.go:37 Error dialing x509: certificate signed by unknown author> Jan 24 18:25:48 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:48.596+0530 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(> Jan 24 18:25:48 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:48.596+0530 INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(elasticse> Jan 24 18:25:48 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:48.596+0530 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer Jan 24 18:25:48 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:48.597+0530 INFO [publisher] pipeline/retry.go:223 done Jan 24 18:25:48 VCPL-LP-93 heartbeat[380760]: 2025-01-24T18:25:48.606+0530 ERROR [esclientleg] transport/logging.go:37 Error dialing x509: certificate signed by unknown author> ``` thank you in advance. ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [January 24, 2025, 1:49pm UTC](https://discuss.elastic.co/t/uptime-heartbeat-service-is-failing/373558/14 "2025-01-24T13:49:27Z")
Something is not OK with the certificate, it's not the same author/issuer. Pay attention which cert you put in which param. You have 3: CA, cert, private key. Temporarily you can set `verification_mode: "none"` . In case you need [the documentation](https://www.elastic.co/guide/en/beats/heartbeat/current/configuration-ssl.html).