Hey! We're using the NGINX Ingress Controller (GitHub - kubernetes/ingress-nginx: NGINX Ingress Controller for Kubernetes), hosted in our Kubernetes cluster. We're using Filebeat deployed as a DaemonSet that parses logs and pushes to our central Elasticsearch.
The Ingress controller logs three different types of logs in the same output stream:
- Access logs
- NGINX error logs
- Ingress Controller logs
I've tried using NGINX module of Filebeat for all three types to read from the same log file, but it does not work very well. Here is my configuration:
logging:
level: info
filebeat:
autodiscover:
providers:
- type: kubernetes
labels:
dedot: true
annotations:
dedot: true
templates:
- condition:
equals:
kubernetes.container.name: "nginx-ingress-controller"
config:
- module: nginx
access:
enabled: true
input:
type: container
format: cri
paths:
- "/var/log/containers/${data.kubernetes.pod.name}_${data.kubernetes.namespace}_${data.kubernetes.container.name}-${data.kubernetes.container.id}.log"
symlinks: true
error:
enabled: true
input:
type: container
format: cri
paths:
- "/var/log/containers/${data.kubernetes.pod.name}_${data.kubernetes.namespace}_${data.kubernetes.container.name}-${data.kubernetes.container.id}.log"
symlinks: true
ingress_controller:
enabled: true
input:
type: container
format: cri
paths:
- "/var/log/containers/${data.kubernetes.pod.name}_${data.kubernetes.namespace}_${data.kubernetes.container.name}-${data.kubernetes.container.id}.log"
symlinks: true
output:
elasticsearch:
hosts:
- 'central-elasticsearch:9200'
Each kind gets errors when it reaches a format that it doesn't support. These three types seem to be supported to only look at different log files. Does anyone have any suggestions on how we can solve this?