Hi! I want to set up a metricbeat data collection for some pods in my cluster. I already have a working filebeat configuration where I collect logs from certain pods based on autodiscover template conditions. But the same configuration does not work for metricbeat.
I took the YAML of the metricbeat setup from the official site. The relevant part that I edited is this:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-daemonset-config
namespace: kube-system
labels:
k8s-app: metricbeat
data:
metricbeat.yml: |-
metricbeat.autodiscover:
providers:
- type: kubernetes
scope: cluster
node: ${NODE_NAME}
unique: true
templates:
- condition:
equals:
kubernetes.labels.test: foo
config:
- module: kubernetes
hosts: ["kube-state-metrics:8080"]
period: 10s
add_metadata: true
metricsets:
- state_node
- state_deployment
- state_daemonset
- state_replicaset
- state_pod
- state_container
- state_job
- state_cronjob
- state_resourcequota
- state_statefulset
- state_service
But with this configuration, no metrics are collected at all. When I remove the - condition node, then metrics are collected but from all the pods in the cluster. When the metrics are collected, I can verify that there are in fact documents from metricbeat from a pod with kubernetes.labels.test: foo. I can filter on those fields and they do exist.
But when I change the config again to include the - condition, then the stream of documents stops, including from the pod with the correct label.
Is this a known issue or is there something wrong with my configuration? Thanks