# How to monitor beats using metricbeat module with autodiscover
**URL:** https://discuss.elastic.co/t/how-to-monitor-beats-using-metricbeat-module-with-autodiscover/306060
**Category:** Elastic Cloud on Kubernetes (ECK)
**Tags:** docker
**Created:** [May 31, 2022, 6:59pm UTC](https://discuss.elastic.co/t/how-to-monitor-beats-using-metricbeat-module-with-autodiscover/306060 "2022-05-31T18:59:15Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Kay\_Khan](https://discuss.elastic.co/u/Kay_Khan)
#### Post date: [May 31, 2022, 6:59pm UTC](https://discuss.elastic.co/t/how-to-monitor-beats-using-metricbeat-module-with-autodiscover/306060/1 "2022-05-31T18:59:15Z")
I have setup metricbeats to monitor Elasticsearch and kibana which work fine.
However i want to to add the module "beats" to monitor filebeat etc. [Beat module | Metricbeat Reference [8.2] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-beat.html)
Note that i set `http.enabled` to true as recommended in the docs: [Configure an HTTP endpoint for metrics | Metricbeat Reference [8.2] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/8.2/http-endpoint.html)
One possible thing which could be wrong is the `host` as ive just copied that from the same as Elasticsearch and kibana module config.
What am i doing wrong here? (if its hosts, what should it be for auto discovery)
**metricbeat.yaml**
```auto
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
name: metricbeat
namespace: elastic-system
spec:
type: metricbeat
version: 8.2.0
elasticsearchRef:
name: elasticsearch-prod
kibanaRef:
name: kibana-prod
config:
http.enabled: true
setup.template.settings:
index:
routing.allocation.require.type: "monitoring"
metricbeat:
autodiscover:
providers:
- type: kubernetes
scope: cluster
hints.enabled: true
templates:
- condition:
contains:
kubernetes.labels.scrape: es
config:
- module: elasticsearch
metricsets:
- ccr
- cluster_stats
- enrich
- index
- index_recovery
- index_summary
- ml_job
- node_stats
- shard
period: 10s
hosts: "https://${data.host}:${data.ports.https}"
username: ${MONITORED_ES_USERNAME}
password: ${MONITORED_ES_PASSWORD}
# WARNING: disables TLS as the default certificate is not valid for the pod FQDN
# TODO: switch this to "certificate" when available: https://github.com/elastic/beats/issues/8164
ssl.verification_mode: "none"
xpack.enabled: true
- condition:
contains:
kubernetes.labels.scrape: kb
config:
- module: kibana
metricsets:
- stats
period: 10s
hosts: "https://${data.host}:${data.ports.https}"
username: ${MONITORED_ES_USERNAME}
password: ${MONITORED_ES_PASSWORD}
# WARNING: disables TLS as the default certificate is not valid for the pod FQDN
# TODO: switch this to "certificate" when available: https://github.com/elastic/beats/issues/8164
ssl.verification_mode: "none"
xpack.enabled: true
- condition:
contains:
kubernetes.labels.common.k8s.elastic.co/type: beat
config:
- module: beat
metricsets:
- stats
- state
period: 60s
hosts: "http://localhost:5066"
username: ${MONITORED_ES_USERNAME}
password: ${MONITORED_ES_PASSWORD}
# WARNING: disables TLS as the default certificate is not valid for the pod FQDN
# TODO: switch this to "certificate" when available: https://github.com/elastic/beats/issues/8164
ssl.verification_mode: "none"
xpack.enabled: true
modules:
- module: system
period: 10s
metricsets:
- cpu
- load
- memory
- network
- process
- process_summary
process:
include_top_n:
by_cpu: 5
by_memory: 5
processes:
- .*
- module: system
period: 1m
metricsets:
- filesystem
- fsstat
processors:
- drop_event:
when:
regexp:
system:
filesystem:
mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib)($|/)
- module: kubernetes
period: 10s
node: ${NODE_NAME}
hosts:
- https://${NODE_NAME}:10250
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
ssl:
verification_mode: none
metricsets:
- node
- system
- pod
- container
- volume
processors:
- add_cloud_metadata: {}
- add_host_metadata: {}
daemonSet:
podTemplate:
spec:
serviceAccountName: metricbeat
automountServiceAccountToken: true # some older Beat versions are depending on this settings presence in k8s context
containers:
- args:
- -e
- -c
- /etc/beat.yml
- -system.hostfs=/hostfs
name: metricbeat
volumeMounts:
- mountPath: /hostfs/sys/fs/cgroup
name: cgroup
- mountPath: /var/run/docker.sock
name: dockersock
- mountPath: /hostfs/proc
name: proc
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: MONITORED_ES_USERNAME
value: elastic
- name: MONITORED_ES_PASSWORD
valueFrom:
secretKeyRef:
key: elastic
name: elasticsearch-prod-es-elastic-user
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: aws-secret
key: access-key-id
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: aws-secret
key: access-secret-key
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true # Allows to provide richer host metadata
securityContext:
runAsUser: 0
terminationGracePeriodSeconds: 30
volumes:
- hostPath:
path: /sys/fs/cgroup
name: cgroup
- hostPath:
path: /var/run/docker.sock
name: dockersock
- hostPath:
path: /proc
name: proc
---
# permissions needed for metricbeat
# source: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-kubernetes.html
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: metricbeat
namespace: elastic-system
rules:
- apiGroups:
- ""
resources:
- nodes
- namespaces
- events
- pods
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- replicasets
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- statefulsets
- deployments
- replicasets
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- nodes/stats
verbs:
- get
- nonResourceURLs:
- /metrics
verbs:
- get
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: metricbeat
namespace: elastic-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: metricbeat
namespace: elastic-system
subjects:
- kind: ServiceAccount
name: metricbeat
namespace: elastic-system
roleRef:
kind: ClusterRole
name: metricbeat
apiGroup: rbac.authorization.k8s.io
---
```
**image showing no discovery of beats**
**beats running**
```auto
Every 2.0s: kubectl get elastic -n elastic-system klaptop: Tue May 31 20:00:13 2022
NAME HEALTH NODES VERSION AGE
apmserver.apm.k8s.elastic.co/apm-server-prod green 1 8.2.0 3h42m
NAME HEALTH NODES VERSION PHASE AGE
elasticsearch.elasticsearch.k8s.elastic.co/elasticsearch-prod green 6 8.2.0 Ready 8d
NAME HEALTH NODES VERSION AGE
kibana.kibana.k8s.elastic.co/kibana-prod green 1 8.2.0 8d
NAME HEALTH AVAILABLE EXPECTED TYPE VERSION AGE
beat.beat.k8s.elastic.co/filebeat green 5 5 filebeat 8.2.0 8d
beat.beat.k8s.elastic.co/metricbeat green 5 5 metricbeat 8.2.0 63m
```
**label attached:**
---
### Author:  [@framsouza](https://discuss.elastic.co/u/framsouza)
#### Post date: [June 2, 2022, 10:04am UTC](https://discuss.elastic.co/t/how-to-monitor-beats-using-metricbeat-module-with-autodiscover/306060/2 "2022-06-02T10:04:29Z")
Perhaps are you talking about this? [enable beats monitoring in ECK · Issue #5563 · elastic/cloud-on-k8s · GitHub](https://github.com/elastic/cloud-on-k8s/issues/5563)
---
### Author:  [@Kay\_Khan](https://discuss.elastic.co/u/Kay_Khan)
#### Post date: [June 6, 2022, 2:48pm UTC](https://discuss.elastic.co/t/how-to-monitor-beats-using-metricbeat-module-with-autodiscover/306060/4 "2022-06-06T14:48:52Z")
I'm not sure what that is about, i am trying to monitor beats (filebeats, metricbeats) using the metricbeat module with autodiscovery
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 4, 2022, 2:49pm UTC](https://discuss.elastic.co/t/how-to-monitor-beats-using-metricbeat-module-with-autodiscover/306060/5 "2022-07-04T14:49:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.