Kubernetes With Filebeat parsing nginx container

I have a nginx based container in kubernetes. It is running and create standard nginx logs to stdout and stderr. These logs needs to be grabbed and send to elasticsearch with nginx module parser.

I used as referrence the following: https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat-kubernetes.yaml

If I use the:

filebeat.inputs:
- type: container
  paths:
    - /var/log/containers/*.log

In this case the logs are send, but without nginx module parser. It sends container specific attributes and messages contains the whole nginx log. Without parsing.

If I try to use autohint with it:

filebeat.autodiscover:
providers:
- type: kubernetes
hints.enabled: true
hints.default_config:
type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log

Then I add under spec and template the annotiation for nginx for the nginx container pod:

spec.template:
co.elastic.logs/module: "nginx"
co.elastic.logs/fileset.stdout: "access"
co.elastic.logs/fileset.stderr: "error"

In this case nothing happen. The elasticsearch won't get anything. Not sure the annotation was not woring or what is the reason.

The goal would be to be able to get the container log, parsed by the nginx module and send it to the elaticsearch.

Can you help me which config definition should I use in filebeat.yml?

Thanks in advance.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.