Filebeat not getting logs from kubernetes pods

Elastic Observability Logs
1

Hi!

I'm trying to use Filebeat on my aws eks to get my containers logs.

filebeat-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: elk
data:
  filebeat.yml: |-
    logging.level: debug
    filebeat.inputs:
    - type: docker
      containers.ids:
        - "*"
      processors:
      - add_kubernetes_metadata:
          in_cluster: true
          namespace: "crm"  # Substitua com o namespace desejado
    output.console:
      pretty: true

filebeat-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: filebeat
  namespace: elk
spec:
  replicas: 1
  selector:
    matchLabels:
      app: filebeat
  template:
    metadata:
      labels:
        app: filebeat
    spec:
      containers:
      - name: filebeat
        image: docker.elastic.co/beats/filebeat:7.10.0
        volumeMounts:
        - name: config
          mountPath: /usr/share/filebeat/filebeat.yml
          subPath: filebeat.yml
          readOnly: true
      volumes:
      - name: config
        configMap:
          name: filebeat-config

Can someone help me? When I've opened the Filebeat logs, it's showing that localize crm pods, but doest not getting log from stdout

2024-01-04T17:09:34.882Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:182 Adding kubernetes pod: crm/crm-base-578f9cc95f-q6htb {"libbeat.processor": "add_kubernetes_metadata"}
2024-01-04T17:09:34.882Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:258 Created index 80cb8c9adf3b33dee263270b36ab4adf873f3824483d7c86b958edf27f4ff3df for pod crm/crm-base-578f9cc95f-q6htb {"libbeat.processor": "add_kubernetes_metadata"}
2024-01-04T17:09:34.882Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:182 Adding kubernetes pod: crm/crm-base-externo-7cdd9b64d7-4nqll {"libbeat.processor": "add_kubernetes_metadata"}
2024-01-04T17:09:34.882Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:258 Created index 768b64af7f5b466bc4b7c6cf0c5374ea4dc359ef73e63dd0a0907ff1029d8dfc for pod crm/crm-base-externo-7cdd9b64d7-4nqll {"libbeat.processor": "add_kubernetes_metadata"}
2

Hi @gustavo_luigi_cev Welcome to the community!

Curious if you tried the recommended configuration to start...

From the docs here

To download the manifest file, run:

curl -L -O https://raw.githubusercontent.com/elastic/beats/7.10/deploy/kubernetes/filebeat-kubernetes.yaml

Also 7.10 is very very old.

3

Hi Stephen! Thanks for your help!

One question, using this yaml, I'll can get stdout logs from my containers?

4

hello gustavo, a couple of things:

  1. as Stephen is suggesting, try to use that filebeat manifest from our repo. it is the most up to date.
  2. the new filebeat manifest should fix some of your issues like replacing filebeat.input = docker. You should use the container input type instead like it is configured in our manifest
  3. you might want to customize that filebeat manifest to your liking though.
    3.1. you need to add logging.level: debug since it is not provided by default
    3.2. by default the new manifest have filebeat running in the namespace kube-system and it has been configured to work only in that namespace. if you want to run into a different namespace you might want to change a couple of things including all references to the namespace but also the role permissions.
    3.3. in our manifest, filebeat is running as a daemonset = 1 instance per node. this is the recommended way to run it. if running as a deployment (like you are doing) with a single replica and you have more than a single node in your k8s cluster you will be missing container logs from other nodes

I hope this helps

5

Oooo, nice!!!

Now I've understood! Thanks for the help, guys! Now it's working!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.