# Heartbeat monitor auto reloading does now work

**URL:** https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306
**Category:** Beats
**Tags:** heartbeat
**Created:** [September 11, 2020, 10:53am UTC](https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306 "2020-09-11T10:53:05Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![tterranigma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tterranigma/32/48360_2.png) [@tterranigma](https://discuss.elastic.co/u/tterranigma)
#### Post date: [September 11, 2020, 10:53am UTC](https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306/1 "2020-09-11T10:53:06Z")

</div>

According to [heartbeat documentation](https://www.elastic.co/guide/en/beats/heartbeat/current/heartbeat-reference-yml.html), heartbeat supports automatically reloading the monitor files periodically. In my installation of heartbeat, I have enabled this:

```
heartbeat:
  config:
    monitors:
      path: /etc/heartbeat/monitors.d//*.yml
    reload:
      enabled: true
      period: 5s

```

However, this does not work and I have to restart the heartbeat service every time I create new monitors or change existing ones.

This is how the heartbeat process looks like:

```
root 9883 14.1 1.1 1146752 97348 ? Ssl 09:55 7:40 /usr/share/heartbeat/bin/heartbeat -environment systemd -e -d elasticsearch -c /etc/heartbeat/heartbeat.yml -path.home /usr/share/heartbeat -path.config /etc/heartbeat -path.data /var/lib/heartbeat -path.logs /var/log/heartbeat

```

The heartbeat monitors have the following permissions:

```
-rw-r--r-- 1 root root 324 Sep 10 11:05 monitors.d/twmn.elasticsearch.yml

```

and the folder containing the monitors:

```
drwxr-xr-x 2 root root 102400 Sep 11 09:55 monitors.d/

```

Am I doing sth wrong?

---

<div class="post-metadata">

### Author: ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)
#### Post date: [September 14, 2020, 1:25pm UTC](https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306/2 "2020-09-14T13:25:22Z")

</div>

Hi, what version of Heartbeat are you using?

Also, could you run Heartbeat with `logging.level: debug` set in your `heartbeat.yml`, then change a monitor or create a new one, and post the Heartbeat logs for the following 10 seconds or so?

Thanks,

Shaunak

---

<div class="post-metadata">

### Author: ![tterranigma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tterranigma/32/48360_2.png) [@tterranigma](https://discuss.elastic.co/u/tterranigma)
#### Post date: [September 15, 2020, 12:01pm UTC](https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306/3 "2020-09-15T12:01:06Z")

</div>

I am using version 7.8.1. I remember it being this way with a few previous versions as well. Here is what journalctl says. I run `journalctl -f` first, then edit a monitor from `enabled: False` to `disabled: True`, wait for half a minute and then restart the service manually to have it loaded. I remove many lines that simply count the events reported.

```auto

Sep 15 11:53:30 earth-kibana heartbeat[2998]: 2020-09-15T11:53:30.111Z INFO cfgfile/reload.go:224 Loading of config files completed.
Sep 15 11:53:30 earth-kibana heartbeat[2998]: 2020-09-15T11:53:30.241Z DEBUG [elasticsearch] elasticsearch/client.go:229 PublishEvents: 50 events have been published to elasticsearch in 108.02437ms.
Sep 15 11:53:33 earth-kibana heartbeat[2998]: 2020-09-15T11:53:33.945Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
Sep 15 11:53:33 earth-kibana heartbeat[2998]: 2020-09-15T11:53:33.985Z DEBUG [elasticsearch] elasticsearch/client.go:229 PublishEvents: 50 events have been published to elasticsearch in 551.713238ms.
Sep 15 11:53:38 earth-kibana heartbeat[2998]: 2020-09-15T11:53:38.930Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
Sep 15 11:53:38 earth-kibana heartbeat[2998]: 2020-09-15T11:53:38.994Z DEBUG [elasticsearch] elasticsearch/client.go:229 PublishEvents: 50 events have been published to elasticsearch in 132.151003ms.
Sep 15 11:53:57 earth-kibana heartbeat[2998]: 2020-09-15T11:53:57.014Z WARN scheduler/scheduler.go:152 15 tasks have missed their schedule deadlines in the last 15s.
Sep 15 11:53:57 earth-kibana systemd[1]: Stopping Ping remote services for availability and log results to Elasticsearch or send to Logstash....
Sep 15 11:53:57 earth-kibana heartbeat[2998]: 2020-09-15T11:53:57.764Z INFO beater/heartbeat.go:120 Shutting down.
Sep 15 11:53:57 earth-kibana heartbeat[2998]: 2020-09-15T11:53:57.808Z INFO cfgfile/reload.go:227 Dynamic config reloader stopped
Sep 15 11:53:57 earth-kibana heartbeat[2998]: 2020-09-15T11:53:57.808Z INFO [reload] cfgfile/list.go:118 Stopping 963 runners ...
Sep 15 11:53:57 earth-kibana heartbeat[2998]: 2020-09-15T11:53:57.863Z INFO [api] api/server.go:66 Stats endpoint (192.168.1.5:5070) finished: accept tcp 192.168.1.5:5070: use of closed network connection
Sep 15 11:53:57 earth-kibana heartbeat[2998]: 2020-09-15T11:53:57.864Z INFO instance/beat.go:469 heartbeat stopped.
Sep 15 11:53:57 earth-kibana systemd[1]: Stopped Ping remote services for availability and log results to Elasticsearch or send to Logstash..
Sep 15 11:53:57 earth-kibana systemd[1]: Started Ping remote services for availability and log results to Elasticsearch or send to Logstash..
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.298Z INFO instance/beat.go:647 Home path: [/usr/share/heartbeat] Config path: [/etc/heartbeat] Data path: [/var/lib/heartbeat] Logs path: [/var/log/heartbeat]
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.308Z INFO instance/beat.go:655 Beat ID: 9805c7c4-6ede-5d01-92a3-2abe0d3960db
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.311Z INFO [api] api/server.go:62 Starting stats endpoint
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.311Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.313Z INFO [beat] instance/beat.go:983 Beat info {"system_info": {"beat": {"path": {"config": "/etc/heartbeat", "data": "/var/lib/heartbeat", "home": "/usr/share/heartbeat", "logs": "/var/log/heartbeat"}, "type": "heartbeat", "uuid": "9805c7c4-6ede-5d01-92a3-2abe0d3960db"}}}
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.314Z INFO [beat] instance/beat.go:992 Build info {"system_info": {"build": {"commit": "94f7632be5d56a7928595da79f4b829ffe123744", "libbeat": "7.8.1", "time": "2020-07-21T14:22:56.000Z", "version": "7.8.1"}}}
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.314Z INFO [api] api/server.go:64 Metrics endpoint listening on: 192.168.1.5:5070 (configured: 192.168.1.5)
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.314Z INFO [beat] instance/beat.go:995 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.13.10"}}}
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.319Z INFO [beat] instance/beat.go:999 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-09-11T20:45:22Z","containerized":false,"name":"earth-kibana","ip":["127.0.0.1/8","::1/128","192.168.1.5/32","fe80::4001:c0ff:fea8:105/64"],"kernel_version":"5.0.0-1028-gcp","mac":["42:01:c0:a8:01:05"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.3 LTS (Bionic Beaver)","major":18,"minor":4,"patch":3,"codename":"bionic"},"timezone":"UTC","timezone_offset_sec":0,"id":"e309c5fbad3714e4eb72cfbf36c8adaa"}}}
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.320Z INFO [beat] instance/beat.go:1028 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/", "exe": "/usr/share/heartbeat/bin/heartbeat", "name": "heartbeat", "pid": 3068, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2020-09-15T11:53:57.570Z"}}}
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.320Z INFO instance/beat.go:310 Setup Beat: heartbeat; Version: 7.8.1
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.320Z INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'heartbeat-7.8.1' as ILM is enabled.
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.355Z INFO eslegclient/connection.go:99 elasticsearch url: https://35.206.159.154:9200
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.355Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.355Z INFO eslegclient/connection.go:99 elasticsearch url: https://35.210.163.90:9200
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.355Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.355Z INFO eslegclient/connection.go:99 elasticsearch url: https://35.210.170.219:9200
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.355Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.356Z INFO [publisher] pipeline/module.go:113 Beat name: earth-kibana-heartbeat
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.356Z INFO instance/beat.go:463 heartbeat start running.
Sep 15 11:53:58 earth-kibana heartbeat[3068]: 2020-09-15T11:53:58.356Z INFO beater/heartbeat.go:82 heartbeat is running! Hit CTRL-C to stop it.
Sep 15 11:54:00 earth-kibana heartbeat[3068]: 2020-09-15T11:54:00.236Z INFO cfgfile/reload.go:164 Config reloader started
Sep 15 11:54:00 earth-kibana heartbeat[3068]: 2020-09-15T11:54:00.466Z ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Monitor not loaded, plugin is disabled
Sep 15 11:54:00 earth-kibana heartbeat[3068]: 2020-09-15T11:54:00.468Z ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Monitor not loaded, plugin is disabled
Sep 15 11:54:00 earth-kibana heartbeat[3068]: 2020-09-15T11:54:00.469Z ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Monitor not loaded, plugin is disabled
(END)

```

---

<div class="post-metadata">

### Author: ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)
#### Post date: [September 15, 2020, 12:37pm UTC](https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306/4 "2020-09-15T12:37:32Z")

</div>

Looking at the last three lines of the logs it looks like, for some reason, Heartbeat doesn't like the monitor configurations. Would you mind posting those here (with any sensitive information redacted)?

Thanks,

Shaunak

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [September 16, 2020, 1:22am UTC](https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306/5 "2020-09-16T01:22:19Z")

</div>

> [@tterranigma](#):
>
> `/etc/heartbeat/monitors.d//*.yml`

Did notice you have an extra `/` in your path probably not the issue but it is extraneous

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 14, 2020, 3:22am UTC](https://discuss.elastic.co/t/heartbeat-monitor-auto-reloading-does-now-work/248306/6 "2020-10-14T03:22:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
