Heartbeat TCP monitor does not return a "connection refused" message

[norgro2601 (Norbert Groene)]

Hi,

we've moved the monitoring of an application from Nagios to Heartbeat. Now we noticed a difference, which is causing some trouble.
A TCP check in Nagios for a specific port returns a "connection refused" as result, the similar check in Heartbeat returns an "up", although the port is not working at that time. The checked URL is the same.
Is there some additional setting I must do in the heartbeat.yml to get this back as "down"?

Kind regards,
Norbert

[stephenb (Stephen Brown)]

Hi @norgro2601

I just tested tcp monitor with a mysql instance running on port 3306.
When mysql is running on 3306 I get UP
When mysql is not running i get DOWN

My very basic tcp config, perhaps you have a typo somewhere? Perhaps you can share your config.

- type: tcp # monitor type `tcp`. Connect via TCP and optionally verify endpoint
  # by sending/receiving a custom payload

  # ID used to uniquely identify this monitor in elasticsearch even if the config changes
  id: my-tcp-monitor

  # Human readable display name for this service in Uptime UI and elsewhere
  name: My TCP monitor

  # Name of corresponding APM service, if Elastic APM is in use for the monitored service.
  #service.name: my-apm-service-name

  # Enable/Disable monitor
  #enabled: true

  # Configure task schedule
  schedule: '@every 5s' # every 5 seconds from start of beat

  # configure hosts to ping.
  # Entries can be:
  #   - plain host name or IP like `localhost`:
  #       Requires ports configs to be checked. If ssl is configured,
  #       a SSL/TLS based connection will be established. Otherwise plain tcp connection
  #       will be established
  #   - hostname + port like `localhost:12345`:
  #       Connect to port on given host. If ssl is configured,
  #       a SSL/TLS based connection will be established. Otherwise plain tcp connection
  #       will be established
  #   - full url syntax. `scheme://<host>:[port]`. The `<scheme>` can be one of
  #     `tcp`, `plain`, `ssl` and `tls`. If `tcp`, `plain` is configured, a plain
  #     tcp connection will be established, even if ssl is configured.
  #     Using `tls`/`ssl`, an SSL connection is established. If no ssl is configured,
  #     system defaults will be used (not supported on windows).
  #     If `port` is missing in url, the ports setting is required.
  hosts: ["localhost:3306"]

#

[norgro2601 (Norbert Groene)]

Hi @stephenb,

this is my configuration.

- type: tcp # monitor type `tcp`. Connect via TCP and optionally verify endpoint
  # by sending/receiving a custom payload

  # ID used to uniquely identify this monitor in elasticsearch even if the config changes
  id: domino-notes-01-domino-port-monitor

  # Monitor name used for job name and document type
  name: Domino Notes-01 Domino Port

  # Enable/Disable monitor
  enabled: true

  # Configure task schedule
  schedule: '@every 5m' # every 5 minutes from start of beat

  # configure hosts to ping.
  # Entries can be:
  #   - plain host name or IP like `localhost`:
  #       Requires ports configs to be checked. If ssl is configured,
  #       a SSL/TLS based connection will be established. Otherwise plain tcp connection
  #       will be established
  #   - hostname + port like `localhost:12345`:
  #       Connect to port on given host. If ssl is configured,
  #       a SSL/TLS based connection will be established. Otherwise plain tcp connection
  #       will be established
  #   - full url syntax. `scheme://<host>:[port]`. The `<scheme>` can be one of
  #     `tcp`, `plain`, `ssl` and `tls`. If `tcp`, `plain` is configured, a plain
  #     tcp connection will be established, even if ssl is configured.
  #     Using `tls`/`ssl`, an SSL connection is established. If no ssl is configured,
  #     system defaults will be used (not supported on windows).
  #     If `port` is missing in url, the ports setting is required.
  hosts: ["notes-01.gis-intern.de:1352"]

It looks like yours, the only difference seems to be, that the check starts from a remote host.

[stephenb (Stephen Brown)]

@norgro2601

"notes-01.gis-intern.de:1352"

Perhaps run with debug on and take a closer look

sudo ./heartbeat -e -d "*"

[Andrew_Cholakian1 (Andrew Cholakian)]

@norgro2601 Have you tested this with a third tool? TCP is fairly basic functionality, and there's not much that Heartbeat does on top of the OS kernel to perform the check. I'd try nc (netcat) or similar to see if other tools can reproduce this behavior.

[Andrew_Cholakian1 (Andrew Cholakian)]

Another thought, is there a firewall behaving differently for Heartbeat than nagios perhaps?

[norgro2601 (Norbert Groene)]

I've tested heartbeat with debug on, but unfortunately this returns no additional information to solve this problem.

[norgro2601 (Norbert Groene)]

I've noticed the same behavior with heartbeat running in different locations with different firewalls, so I don't think this is caused by the firewall.

[norgro2601 (Norbert Groene)]

I will give this tool a try.