Enabling nginx module in filebeat on docker

[g.myznikov.tinkoff (Gleb)]

Hello!
I have run filebeat on docker by a custom image. Its Dockerfile is:

FROM docker.elastic.co/beats/filebeat:6.3.1
COPY filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
RUN filebeat modules enable nginx
RUN filebeat setup

It is obviously that I try to enable nginx module. http://localhost:5601/app/kibana#/home/tutorial/nginxLogs?_g=() is my tutorial. Starting docker conteiner seems to work but nginx module stays disable. Any thoughts about what I've done wrong.
P.S.
My filebeat.yml:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /testvar/*log
    - /testvar/**/*log

filebeat.config:
  modules:
    path: /usr/share/filebeat/modules.d/*.yml


output.elasticsearch:
  hosts: ["localhost:9200"]

setup.kibana:
  host: "localhost:5601"

[jsoriano (Jaime Soriano)]

Hi @g.myznikov.tinkoff,

You shouldn't need to create a custom image. You can mount the configuration files on runtime with the --volume or --mount flags of docker run, this way you can provide a custom filebeat.yml, and also additional modules configuration, what could include modules enabling.

filebeat setup should be run only once, from a machine with access to elasticsearch and kibana, take into account that RUN commands in Dockerfiles are only run when the image is built.

Other thing you can try when using docker is autodiscover and hints-based configuration, these features allow to enable and configure modules dynamically depending on a set of rules and the container settings, so you don't need to include static configurations in your configuration files.

[g.myznikov.tinkoff (Gleb)]

Thanks for your answer. I've used your advice about including all information about modules in filebeat.yml. And now everthing seems to work fine. My filebeat.yml is:

filebeat.modules:
- module: nginx
  access:
    var.paths: ["/testvar/nginx/access.log"]
  error:
    var.paths: ["/testvar/nginx/error.log"]

output.elasticsearch:
  hosts: ["localhost:9200"]

setup.kibana:
  host: "localhost:5601"

And command for starting conteiner:

docker run -it --user=root -v /dockerfilebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml -v /var/log:/testvar docker.elastic.co/beats/filebeat:6.3.1