K8S metadata is missing

Elastic Stack Beats
1

Hi there!
I'm using ELK 7.3.1
And when Filebeat collects logs from Kubernetes, I don't see any Kubernetes metadata (namespace is missing, container, name and etc.). What am I doing wrong here?
Attaching screen of appeared data -

image
image.png1014×521 53.4 KB
Screen Shot 2019-09-09 at 12.22.02

My filebeat config -

apiVersion: v1                 
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/pods/*/*/*.log
      containers.ids:
            - "*"
      message_key: log
      keys_under_root: true
      symlinks: true
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
      scan_frequency: 10s
      close_inactive: 1m
    output.elasticsearch:
      hosts: ${ELASTICSEARCH_HOST:?No logstash host configured. Use env var ELASTICSEARCH_HOST to set hosts.}

apiVersion: v1         
kind: ConfigMap
metadata:
  name: filebeat-inputs
  namespace: kube-system
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  kubernetes.yml: |-
    filebeat.inputs:
    - type: container
      containers.paths:
        -  '/var/log/pods/*/*/*.log'
      message_key: log
      keys_under_root: true
      symlinks: true
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
      scan_frequency: 10s
      close_inactive: 1m

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
  name: filebeat
  namespace: kube-system
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: filebeat
      kubernetes.io/cluster-service: "true"
  template:
    metadata:
      labels:
        k8s-app: filebeat
        kubernetes.io/cluster-service: "true"
    spec:
      containers:
      - args:
        - -c
        - /etc/filebeat.yml
        - -e
        env:
        - name: ELASTICSEARCH_HOST
          value: http://........:9200 .   //LB
        image: docker.elastic.co/beats/filebeat:7.3.1
        imagePullPolicy: IfNotPresent
        name: filebeat
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 100Mi
        securityContext:
          procMount: Default
          runAsUser: 0
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/filebeat.yml
          name: config
          readOnly: true
          subPath: filebeat.yml
        - mountPath: /usr/share/filebeat/inputs.d
          name: inputs
          readOnly: true
        - mountPath: /usr/share/filebeat/data
          name: data
        - mountPath: /var/lib/docker/containers
          name: varlibdockercontainers
          readOnly: true
        - mountPath: /var/log
          name: varlog
          readOnly: true
        - mountPath: /var/data
          name: vardata
          readOnly: true
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: filebeat
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 384
          name: filebeat-config
        name: config
      - hostPath:
          path: /var/log
          type: ""
        name: varlog
      - hostPath:
          path: /var/data
          type: ""
        name: vardata
      - hostPath:
          path: /var/log/pods
          type: ""
        name: varlibdockercontainers
      - configMap:
          defaultMode: 384
          name: filebeat-inputs
        name: inputs
      - emptyDir: {}
        name: data

Please help.

Aleksei

2

Resolved the issue by putting host: ${NODE_NAME} to add_kubernetes_metadata:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.