No Kibana Dashboards in Saved Objects/Dashboards with correct metric beat settings

[cgutshall]

I have a single node EC2 Instance running both Elasticsearch and Kibana, which I am sending kubernetes metrics to from a daemonset running a metricbeat pod for each node In the cluster.

Even with the following settings in my metricbeat.yml:

setup.dashboards.enabled: true
setup.kibana:
host: "http://ec2_internal_ip:5601"

I am unable to see any of the dashboards that should come with metricbeat. This is version 7.5.1 of metricbeat. I am unsure of where to look next to resolve this.

Thanks

[Kaiyan_Sheng]

Did you run ./metricbeat setup --dashboards to import the existing dashboards?

[cgutshall]

I haven't in this case, I'll try adding that to the deploy config to see if that resolves it.

[Kaiyan_Sheng]

Sounds good, thanks!!

[cgutshall]

Since I'm doing my deployment of metricbeat with a Kubernetes Definition file, will I need to update the image or is there a way to run the command you mentioned from that file. This is the piece I'm refereeing too:

#Deploy a Metricbeat instance per node for node metrics retrieval
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: metricbeat
namespace: kube-system
labels:
k8s-app: metricbeat
spec:
selector:
matchLabels:
k8s-app: metricbeat
template:
metadata:
labels:
k8s-app: metricbeat
spec:
serviceAccountName: metricbeat
terminationGracePeriodSeconds: 30
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: metricbeat
image: docker.elastic.co/beats/metricbeat:7.5.1
args: [
"-c", "/etc/metricbeat.yml",
"-e",
"-system.hostfs=/hostfs",
]
# command: ["/bin/sh"]
# args: ["-c", "./metricbeat setup --dashboards"]
env:
- name: ELASTICSEARCH_HOST
value: http://internal_ec2_ip
- name: ELASTICSEARCH_PORT
value: "9200"
- name: KIBANA_PORT
value: "5601"
- name: HOST_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
securityContext:
runAsUser: 0
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 100Mi
volumeMounts:
- name: config
mountPath: /etc/metricbeat.yml
readOnly: true
subPath: metricbeat.yml
- name: modules
mountPath: /usr/share/metricbeat/modules.d
readOnly: true
- name: dockersock
mountPath: /var/run/docker.sock
- name: proc
mountPath: /hostfs/proc
readOnly: true
- name: cgroup
mountPath: /hostfs/sys/fs/cgroup
readOnly: true
volumes:
- name: proc
hostPath:
path: /proc
- name: cgroup
hostPath:
path: /sys/fs/cgroup
- name: dockersock
hostPath:
path: /var/run/docker.sock
- name: config
configMap:
defaultMode: 0600
name: metricbeat-daemonset-config
- name: modules
configMap:
defaultMode: 0600
name: metricbeat-daemonset-modules
- name: data
hostPath:
path: /var/lib/metricbeat-data
type: DirectoryOrCreate

Thanks