Applying docker input to Nginx module with autodiscover

[Claude_Juif]

Hi people,

I'm currently struggling to apply docker input to the Nginx module with filebeat 6.4.1.

Here is my filebeat configuration file :

filebeat.autodiscover:
  providers:
    - type: docker
      templates:
        - condition.contains:
             docker.container.image: nginx
          config:
            - module: nginx
              access:
                enabled: true
                input:
                  - type: docker
                    combine_partial: false
                    fields_under_root: true
                    containers:
                      stream: "stdout"
                      paths:
                        - "/var/lib/docker/containers"
                      ids:
                        - "${data.docker.container.id}"
                    processors:
                      - add_cloud_metadata: ~
                      - add_docker_metadata: ~
                      - decode_json_fields:
                        fields: ['message']
                        target: "json"

If I try to only set the docker inputs, it works but without nginx data, if i try to set only the nginx module, it works, but with a grok parser failure because the nginx log message is a json and not a string.

If I try to mix both of them, nothing happen, no docker logs are sent to ES at all.

It seems I am pretty close to it, but can't figure what's wrong.

I run filebeat with filebeat -e -d '*' but can't see errors.

Here is the attached logs :
Filebeat debug log

Can you please point me in the right direction ?

Best regards,

[Claude_Juif]

Hi,

Sorry for disturbing, but I managed to get it work with this config file.

I left it here in case it can help someone else :

filebeat.autodiscover:
  providers:
    - type: docker
      processors:
      - add_cloud_metadata: ~
      templates:
        - condition:
            contains.docker.container.image: "nginx"
          config:
            - module: nginx
              access:
                input:
                  type: docker
                  processors:
                  - add_cloud_metadata: ~
                  containers.ids: "${data.docker.container.id}"
                  containers.stream: stdout
              error:
                input:
                  type: docker
                  processors:
                  - add_cloud_metadata: ~
                  containers.ids: "${data.docker.container.id}"
                  containers.stream: stderr

Best Regards,