My Filebeat configuration can't listen to my logs inside my Kubernetes pods

I have web app service that run in go and produce a log that I stored inside /var/logs/app/app.log. This service is running inside a Kubernetes pods. And I have another pods that running Filebeat to listen to logs produced by app service. But my Filebeat doesn't seems can listen to the logs produced by apps.

This is my config map for filebeat deployment

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          templates:
            - config:
                - type: container
                  paths:
                    - /var/logs/app/*.log                  

    processors:
      - decode_json_fields:
          fields: ['message']
          target: json
      - rename:
          fields:
            - from: "url"
              to: "url_data"
          ignore_missing: false
          fail_on_error: true

    cloud.id: ${CLOUD_ID}
    cloud.auth : ${CLOUD_AUTH}

And this is my deployment config

apiVersion: apps/v1
kind: Deployment
metadata:
  name: filebeat
  labels:
    k8s-app: filebeat
spec:
  selector:
    matchLabels:
      k8s-app: filebeat
  template:
    metadata:
      labels:
        k8s-app: filebeat
    spec:
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      containers:
        - name: filebeat
          image: docker.elastic.co/beats/filebeat:8.8.1
          args: [
            "-c", "/etc/filebeat.yml",
            "-e",
          ]
          env:
            - name: CLOUD_ID
              valueFrom:
                secretKeyRef:
                  name: filebeat-secret
                  key: cloud.id
            - name: CLOUD_AUTH
              valueFrom:
                secretKeyRef:
                  name: filebeat-secret
                  key: cloud.auth
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          securityContext:
            runAsUser: 0
          resources:
            limits:
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
            - name: config
              mountPath: /etc/filebeat.yml
              readOnly: true
              subPath: filebeat.yml
      volumes:
        - name: config
          configMap:
            defaultMode: 0640
            name: filebeat-config

I expect filebeat can listen custom log instead of docker logs. Where did I go wrong that filebeat can't listen to my app logs.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.