Filebeat Kubernetes autodiscover with post "processor" specific field with another filebeat module

Elastic Stack Beats
5

Thanks all, for the recommendation and suggestion.

Finally the pod/container logs from specific log format has been perfectly parsed using Hints based autodiscover | Filebeat Reference [8.5] | Elastic.

image
image1843×989 145 KB

image
image1832×983 255 KB

In the first time read the docs, I don't get the point on how to put the "hint" that parsed the log from kubernetes pod/container with different module.
The key are "annotations" (for kubernetes) or "labels" (for container/docker/podman etc)
In my case, on kubernetes, every pod/container log that need to be processed by the filebeat module/pipeline, So I need add some annotations. For Deployment we can create sample manifest like this:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp-deployment
spec:
  selector:
    matchLabels:
      app: webapp
  replicas: 2
  template:
    metadata:
      labels:
        app: webapp
      annotations:
        co.elastic.logs/module: nginx
        co.elastic.logs/fileset.stdout: access
        co.elastic.logs/fileset.stderr: error
    spec:
      containers:
      - name: webapp
        image: nginx:1.22.1
        ports:
        - containerPort: 80

Hope that help someone out there! :smiley:

1 Like