Using Filebeat on K8s - Trying to obtain App logs inside Pods

Elastic Stack Beats
1

Hello Team!

We are using Filbeat on Kubernetes - OCP configuration. Currently we have success on obtaining logs through Filebeat, but these only show what you would find by navigating on OCP 4.X Web console...
Pods > "specific pod" > Logs

However, the Application team have set an Application log to temp path inside pod ( /web/server_root_www/tmp/.../ERR.log ).
What would be the best approach for Filebeat to also collect that log ?

Any help or information is appreciated, thank you!

Below is our current configuration that picks up the container logs. Followed guide from this link:
https://www.elastic.co/guide/en/beats/filebeat/master/running-on-kubernetes.html

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          node: ${NODE_NAME}
          hints.enabled: true
          hints.default_config:
            type: container
            paths:
              - /var/log/containers/*${data.kubernetes.container.id}.log
            logging.level: info
2

@solijam Please see below:

Basically dont use a demon set as as you wont access individual pods logs, use a persistent volume to store all logs then mount that within each pod.

Then 1 instance of filebeats to read the persistent volumn files.

Above should work - but im getting an error with file timeout not reading new inputs - hopefully somebody will point us to right direction

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.