Heartbeat autodiscover kubernetes pods

[jamesp220291 (james)]

Hi

i want heartbeat to find every single kubernetes pod and report it to elasticsearch uptime, however all i am able to currentlty get is the kubernetes nodes and my 1x http monitor.

here is my config

data:
  heartbeat.yml: |-
    heartbeat.autodiscover:
      # Autodiscover pods
      providers:
        - type: kubernetes
          resource: pod
          scope: cluster
          hints.enabled: true
          templates:
              config:
                - type: http
                  hosts: ["$${data.host}:$${data.port}"]
                  schedule: "@every 1s"
                  timeout: 1s
    
      # Autodiscover services
      providers:
        - type: kubernetes
          resource: service
          scope: cluster
          node: $${NODE_NAME}
          hints.enabled: true
    
      # Autodiscover nodes
      providers:
        - type: kubernetes
          resource: node
          node: $${NODE_NAME}
          scope: cluster
          templates:
            # Example, check SSH port of all cluster nodes:
            - condition: ~
              config:
                - hosts:
                    - $${data.host}:22
                  name: $${data.kubernetes.node.name}
                  schedule: '@every 10s'
                  timeout: 5s
                  type: tcp

    processors:
      - add_cloud_metadata:


    output.elasticsearch:
      hosts: ['${ES_HOST}']
      username: "elastic"
      password: ${ES_PASSWORD}
      
  
    heartbeat.monitors:
    - type: http
      schedule: '@every 5s'
      urls: ["http://www.google.com"]
      service.name: google
      id: google
      name: google

can anyone see where i am going wrong with the autodiscover resource type pod? the autodiscover node works, but not pods

[jkambic (Justin Kambic)]

Hi @jamesp220291, sorry you're having trouble using autodiscover.

Before proceeding with further troubleshooting I just wanted to verify that the lines like below with the double $ are intentional:

hosts: ["$${data.host}:$${data.port}"]

[jamesp220291 (james)]

hey

yeh thats intentional as its a terraform template the double escaping stop its looking for it as a terraform variable.

[jamesp220291 (james)]

hey @jkambic

do you have any ideas?

[jsoriano (Jaime Soriano)]

Hey @jamesp220291,

You have multiple providers entries under heartbeat.autodiscover, one is probably overriding the others and this is probably why you see that only one provider works.
Try removing all the providers: lines except the first one, so your config looks like this:

    heartbeat.autodiscover:
      providers:
        # Autodiscover pods
        - type: kubernetes
          resource: pod
          ...
        # Autodiscover services
        - type: kubernetes
          resource: service
          ...
        # Autodiscover nodes
        - type: kubernetes
          resource: node
          ...

YAML is sometimes treacherous