Uptime Monitors not reporting/counted correctly

[adam.westbrook]

Kibana v 7.16.2

Our Uptime > Monitors page doesn't seem to be correctly displaying the current status of monitors correctly. We have a single monitor so far, but Uptime is showing "0 Monitors" in the summary. There are also issues with Downtime History being blank, and no monitors being being captured in a basic "Uptime monitor status" rule. Are these issues likely related?

heartbeat.monitors:
  - type: http
    id: REDACTED
    name: REDACTED
    schedule: '@every 5s'
    hosts: ["REDACTED:443"]
    ssl:
      verification_mode: full
      certificate_authorities: ["/etc/ssl/certs/lid.crt"]

output.logstash:
  hosts: ["${logstash_url}"]

logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/heartbeat
  name: heartbeat
  keepfiles: 7
  permissions: 0644

[Andrew_Cholakian1 (Andrew Cholakian)]

Can you make sure the clocks on your Elasticsearch and heartbeat instances are both synchronized?

We use time in some complex ways in the queries powering that page, and there have been different instances of irregularities when heartbeat and Elasticsearch aren't properly synced to NTP, especially when it's off by say 30s or a minute. While I know it seems like a bug there are some performance tradeoffs done by the queries on this page that can result in reduced accuracy when clocks are out of sync.

Another question, is it always '0', or only sometimes '0'?

[adam.westbrook]

Thanks for the quick reply @Andrew_Cholakian1 !

We are using Elastic Cloud so not sure if I'm able to check this? Our Heartbeat instance is using UTC and NTP is synced.

Both the Monitors page and rule creation flyout always say 0.

[shahzad31 (Shahzad)]

Could this be because of Logstash, does removing logstash in between solves this?

[Andrew_Cholakian1 (Andrew Cholakian)]

Hmmmm, it's possible that if logstash is backlogged or adding excessive latency to indexing the docs this could happen.

[Andrew_Cholakian1 (Andrew Cholakian)]

I'll add that adding logstash usually doesn't add much practical benefit for the Uptime use case, and just adds complexity. I'd recommend removing it generally.