Does K8s autodiscover work for all namespaces?

Elastic Stack Beats
1

Hello,

As the title says I was wondering whether hint-based autodiscover for K8s (following this documentation) searches through all namespaces by default?

I have a metricbeat Daemonset running in the kube-system namespace, and am trying to scrape application logs using the following annotations:

co.elastic.metrics/module: prometheus
co.elastic.metrics/metricsets: collector
co.elastic.metrics/hosts: '${data.host}:8080'
co.elastic.metrics/period: 10

Metricbeat is able to ship the K8s cluster metrics but doesn't seem to be shipping the application metrics, which Prometheus running on its own is able to scrape.

2

Hi @ben.sharp,

Yes, autodiscover watches all namespaces by default.

Could you share the autodiscover configuration of your Metricbeat?
Could you also check if Metricbeat is logging any error?

3

I can post the full Yaml configuration file that I am using, but specifically I have the ConfigMap below which from documentation I understood to be where to put the "Metricbeat.autodiscover" options.

---

apiVersion: v1

kind: ConfigMap

metadata:

  name: metricbeat-deployment-modules

  namespace: kube-system

  labels:

    k8s-app: metricbeat

data:

  kubernetes.yml: |-

    - module: kubernetes

      metricsets:

        - state_node

        - state_deployment

        - state_replicaset

        - state_pod

        - state_container

        - event

      period: 10s

      hosts: ["kube-state-metrics.${KUBE_STATE_METRICS_NAMESPACE}:${KUBE_STATE_METRICS_PORT}"]

      enabled: true

      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token

    metricbeat.autodiscover:

      providers:

        - type: kubernetes

          hints.enabled: true

I don't believe it is logging any errors, Metricbeat is already scraping some K8s metrics from the cluster it's running in without issue.

4

Autodiscover configuration has to be placed in the main metricbeat.yml configuration file, not in kubernetes.yml.

You can find a reference manifest for metricbeat in Kubernetes here: https://www.elastic.co/guide/en/beats/metricbeat/7.6/running-on-kubernetes.html

There you can see where the autodiscover configuration is placed.

5

Thanks, I will give that a go.