Heartbeat using Docker hints

Elastic Stack Beats
1

I am trying to setup heartbeat using hints/labels and I am getting the following error

```
2020-04-11T06:02:38.483Z	WARN	[hints.builder]	hints/monitors.go:195	unable to frame a host from input host: %shost.my.domain.io

ERROR	[autodiscover]	autodiscover/autodiscover.go:204	Auto discover config check failed for config '{
"name": "Prom",
"schedule": "@every 10s",
"type": "http"
}', won't start runner: job err hosts is a mandatory parameter accessing config

my docker-compose

    labels:
      #
      # heartbeat
      #
      co.elastic.monitor/1.name: "Prom"
      co.elastic.monitor/1.type: http
      co.elastic.monitor/1.hosts: ${PROM_HOST}
      co.elastic.monitor/1.schedule: '@every 10s'

Heartbeat version 7.6.2

2

Have you verified that ${PROM_HOST} is being set? Also, which version of Heartbeat are you using? Prior to 7.5.0 you would need to set urls, not hosts for HTTP monitors.

3

@Andrew_Cholakian1 Thanks for your reply. I am using version 7.6.2 of Heartbeat. If you see the logs I quoted in this message you can see that the host is not empty. The logs are from the Docker container. In this example it is host.my.domain.io and if I change the ${PROM_HOST} value the change reflects in the Docker logs. Please help. :slight_smile:

4

Thanks for the update. I'm taking a look at this, the root is the code here. I'm thinking that the issue is that we check that ${data.port} is included in the hosts field. I'm not sure why we decided to put that in there. If nothing else, we need a better error message.

You could replace ${PROM_HOST} with ${PROM_HOST}:${data.port} to work around this and ship a fix. Is there a reason you can't use ${data.host}:${data.port} by the way?

5

I've opened https://github.com/elastic/beats/issues/17771 to investigate / track this. @nedim thanks so much for posting here, let's see what we can do to improve this experience.

6

@Andrew_Cholakian1 I do not understand the concept of variables ${data.host}:${data.port} as they are empty in my case whenever I use them. What am I missing?

I subscribed to the github issue. Thanks.

7

@Andrew_Cholakian1 if I use data.host in docker-compose.yml like this:

      co.elastic.monitor/1.name: "Prom"
      co.elastic.monitor/1.type: http
      co.elastic.monitor/1.hosts: "${data.host}:${data.port}"
      co.elastic.monitor/1.schedule: '@every 10s'

I get this error

Invalid interpolation format for "labels" option in service "proxy": "${data.host}:${data.port}"
Invalid interpolation format for "labels" option in service "proxy": "${data.host}:${data.port}"
Invalid interpolation format for "labels" option in service "proxy": "${data.host}:${data.port}"

Testing:

docker-compose -f docker-compose.yml config
ERROR: Invalid interpolation format for "labels" option in service "proxy": "${data.host}:${data.port}"

Per this link https://www.elastic.co/guide/en/beats/heartbeat/current/configuration-autodiscover-hints.html

example is

LABEL co.elastic.monitor/1.type=tcp co.elastic.monitor/1.hosts='${data.host}:6379' co.elastic.monitor/1.schedule='@every 10s'
LABEL co.elastic.monitor/2.type=icmp co.elastic.monitor/2.hosts='${data.host}' co.elastic.monitor/2.schedule='@every 10s'

so my example should work.

Or maybe the example from the documentation is for Dockerfile only and I am using docker-compose which creates the problem?

8

That error comes from python, and looks to be coming from docker compose. It seems some escaping may be missing. A google search turned up Invalid interpolation format for "command" option in service "node-exporter": "^/(sys|proc|dev|host|etc)($|/)" · Issue #4485 · docker/compose · GitHub. I think the solution is to use co.elastic.monitor/1.hosts: "$${data.host}:$${data.port}" in your docker compose file. The $$ is needed in compose.yml AFAICT, but I'm not a docker expert.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.