Grok error with filebeat's nginx module while parsing default nginx logs

Hi, I am trying to setup filebeat in a kubernetes cluster. I only want to gather logs from specific pods.

I decided to run filebeat as daemonset and use the autodiscover and hints. So I can decide by setting pod annotations which contianers I want to monitor in the elastic stack.

My filebeat config looks like this:

    - type: kubernetes
      hints.enabled: true
        enabled: false
        type: container
          - /var/log/containers/*-${}.log  # CRI path

  host: '${NODE_NAME}'
  hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'

We have have an nginx container running. We have no custom logging config, we just copy the website into nginx container during the docker build - using the default config of the nginx container.

The pod has the following annotations:

apiVersion: v1
kind: Pod
  annotations: myapp myenv REMOVED REMOVED
    co.elastic.logs/enabled: "true"
    co.elastic.logs/module: nginx global-unrestricted-psp "false"

But in Kibana I see that all events have not been parsed correctly.
The value of the field error.message is this:

Provided Grok expressions do not match field value: [ - - [13/Jan/2022:14:28:20 +0100] \"GET / HTTP/1.1\" 200 536 \"-\" \"kube-probe/1.19\" \"-\"\n192.168.1.1 - - [13/Jan/2022:14:28:21 +0100] \"GET / HTTP/1.1\" 200 536 \"-\" \"kube-probe/1.19\" \"-\"]

What do I need to do to parse the default nginx logs successfully?
Later I also want to parse the ingress-nginx controller of kubernetes to, but that will be next step.

Thanks in advance,

found it. I also need the following annotations:

co.elastic.logs/fileset.stdout: access
co.elastic.logs/fileset.stderr: error

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.