I have a nginx based container in kubernetes. It is running and create standard nginx logs to stdout and stderr. These logs needs to be grabbed and send to elasticsearch with nginx module parser.
I used as referrence the following: https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat-kubernetes.yaml
If I use the:
filebeat.inputs:
- type: container
paths:
- /var/log/containers/*.log
In this case the logs are send, but without nginx module parser. It sends container specific attributes and messages contains the whole nginx log. Without parsing.
If I try to use autohint with it:
filebeat.autodiscover:
providers:
- type: kubernetes
hints.enabled: true
hints.default_config:
type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
Then I add under spec and template the annotiation for nginx for the nginx container pod:
spec.template:
co.elastic.logs/module: "nginx"
co.elastic.logs/fileset.stdout: "access"
co.elastic.logs/fileset.stderr: "error"
In this case nothing happen. The elasticsearch won't get anything. Not sure the annotation was not woring or what is the reason.
The goal would be to be able to get the container log, parsed by the nginx module and send it to the elaticsearch.
Can you help me which config definition should I use in filebeat.yml?
Thanks in advance.