Hi there!
I'm using ELK 7.3.1
And when Filebeat collects logs from Kubernetes, I don't see any Kubernetes metadata (namespace is missing, container, name and etc.). What am I doing wrong here?
Attaching screen of appeared data -
My filebeat config -
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
namespace: kube-system
labels:
k8s-app: filebeat
data:
filebeat.yml: |-
filebeat.inputs:
- type: container
paths:
- /var/log/pods/*/*/*.log
containers.ids:
- "*"
message_key: log
keys_under_root: true
symlinks: true
processors:
- add_kubernetes_metadata:
in_cluster: true
scan_frequency: 10s
close_inactive: 1m
output.elasticsearch:
hosts: ${ELASTICSEARCH_HOST:?No logstash host configured. Use env var ELASTICSEARCH_HOST to set hosts.}
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-inputs
namespace: kube-system
labels:
k8s-app: filebeat
kubernetes.io/cluster-service: "true"
data:
kubernetes.yml: |-
filebeat.inputs:
- type: container
containers.paths:
- '/var/log/pods/*/*/*.log'
message_key: log
keys_under_root: true
symlinks: true
processors:
- add_kubernetes_metadata:
in_cluster: true
scan_frequency: 10s
close_inactive: 1m
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
labels:
k8s-app: filebeat
kubernetes.io/cluster-service: "true"
name: filebeat
namespace: kube-system
spec:
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: filebeat
kubernetes.io/cluster-service: "true"
template:
metadata:
labels:
k8s-app: filebeat
kubernetes.io/cluster-service: "true"
spec:
containers:
- args:
- -c
- /etc/filebeat.yml
- -e
env:
- name: ELASTICSEARCH_HOST
value: http://........:9200 . //LB
image: docker.elastic.co/beats/filebeat:7.3.1
imagePullPolicy: IfNotPresent
name: filebeat
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 100Mi
securityContext:
procMount: Default
runAsUser: 0
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/filebeat.yml
name: config
readOnly: true
subPath: filebeat.yml
- mountPath: /usr/share/filebeat/inputs.d
name: inputs
readOnly: true
- mountPath: /usr/share/filebeat/data
name: data
- mountPath: /var/lib/docker/containers
name: varlibdockercontainers
readOnly: true
- mountPath: /var/log
name: varlog
readOnly: true
- mountPath: /var/data
name: vardata
readOnly: true
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: filebeat
serviceAccountName: filebeat
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 384
name: filebeat-config
name: config
- hostPath:
path: /var/log
type: ""
name: varlog
- hostPath:
path: /var/data
type: ""
name: vardata
- hostPath:
path: /var/log/pods
type: ""
name: varlibdockercontainers
- configMap:
defaultMode: 384
name: filebeat-inputs
name: inputs
- emptyDir: {}
name: data
Please help.
Aleksei