Docker+filebeat cannot get logs

[qiao_tyrant]

I am using filebeat 6.3.0. I use the official Dockerfile to create a filebeat docker.

But when I run this container, [docker run xxx/custom_image] It only returns
[monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":60,"time":{"ms":63}},"total":{"ticks":100,"time":{"ms":108},"value":100},"user":{"ticks":40,"time":{"ms":45}}},"info":{"ephemeral_id":"1e0835f6-cfcc-437f-975c-af5d4196d345","uptime":{"ms":30043}},"memstats":{"gc_next":4473924,"memory_alloc":3046608,"memory_total":3046608,"rss":12677120}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"kafka"},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0},"writes":1},"system":{"cpu":{"cores":2},"load":{"1":0.01,"15":0.08,"5":0.08,"norm":{"1":0.005,"15":0.04,"5":0.04}}}}}}

And this ticking every 30s, I try to add many logs to the log folder, but It can't work.

And If I choose the local filebeat to work, It can work successfully. By the command service filebeat start.

[pierhugues (Pier-Hugues Pellerin)]

Are you using the official filebeat docker image?

Did you configure some prospectors? We should see them in the log at the beginning.

[qiao_tyrant]

Hi, pierhugues. Yes, I'm using the official docker image.
And my filebeat.yml is configured by below.

  filebeat.inputs:
  - type: log
  enabled: true
  paths:
    - /root/elk-demo/logs/*.log
  fields:
    log_topics: demo3
  output.kafka:
      enabled: true
      hosts: ["localhost:9092","localhost:9093","localhost:9094"]

      worker: 3

      #发送到kafka的topics
      #topic: '%{[fields][log_topics]}'
      topic: 'demo3'
      #broker event partition strategy [random,round_robin,hash] default->hash
      partition.round_robin:
        reachable_only: false

      required_acks: 1
      compression: gzip
      max_message_bytes: 1000000

And my Dockerfile:

FROM docker.elastic.co/beats/filebeat:6.3.0
COPY ./filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
RUN chown filebeat /usr/share/filebeat/filebeat.yml
USER filebeat

And when I start docker's filebeat, It can see the filebeat.yml work, It has the input, but I can't understand why there are no any logs but the service filebeat start can get them.

[FatalGlitch (Tom Callahan)]

/root/elk-demo/logs/*.log is not exposed between your base filesystem and the docker internal filesystem. Use the below run command

docker run -v /root/elk-demo/logs/:/root/elk-demo/logs/ --name filebeat xxx/custom_image

This will map the /root/elk-demo/logs/ directory into the docker container so the filebeat process can read the files from that directory.

[qiao_tyrant]

Thanks, I got it.