Docker+filebeat cannot get logs

I am using filebeat 6.3.0. I use the official Dockerfile to create a filebeat docker.

But when I run this container, [docker run xxx/custom_image] It only returns
[monitoring] log/log.go:124 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":60,"time":{"ms":63}},"total":{"ticks":100,"time":{"ms":108},"value":100},"user":{"ticks":40,"time":{"ms":45}}},"info":{"ephemeral_id":"1e0835f6-cfcc-437f-975c-af5d4196d345","uptime":{"ms":30043}},"memstats":{"gc_next":4473924,"memory_alloc":3046608,"memory_total":3046608,"rss":12677120}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"kafka"},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0},"writes":1},"system":{"cpu":{"cores":2},"load":{"1":0.01,"15":0.08,"5":0.08,"norm":{"1":0.005,"15":0.04,"5":0.04}}}}}}

And this ticking every 30s, I try to add many logs to the log folder, but It can't work.

And If I choose the local filebeat to work, It can work successfully. By the command service filebeat start.

Are you using the official filebeat docker image?

Did you configure some prospectors? We should see them in the log at the beginning.

Hi, pierhugues. Yes, I'm using the official docker image.
And my filebeat.yml is configured by below.

  filebeat.inputs:
  - type: log
  enabled: true
  paths:
    - /root/elk-demo/logs/*.log
  fields:
    log_topics: demo3
  output.kafka:
      enabled: true
      hosts: ["localhost:9092","localhost:9093","localhost:9094"]

      worker: 3

      #发送到kafka的topics
      #topic: '%{[fields][log_topics]}'
      topic: 'demo3'
      #broker event partition strategy [random,round_robin,hash] default->hash
      partition.round_robin:
        reachable_only: false

      required_acks: 1
      compression: gzip
      max_message_bytes: 1000000

And my Dockerfile:

FROM docker.elastic.co/beats/filebeat:6.3.0
COPY ./filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
RUN chown filebeat /usr/share/filebeat/filebeat.yml
USER filebeat

And when I start docker's filebeat, It can see the filebeat.yml work, It has the input, but I can't understand why there are no any logs but the service filebeat start can get them.

/root/elk-demo/logs/*.log is not exposed between your base filesystem and the docker internal filesystem. Use the below run command

docker run -v /root/elk-demo/logs/:/root/elk-demo/logs/ --name filebeat xxx/custom_image

This will map the /root/elk-demo/logs/ directory into the docker container so the filebeat process can read the files from that directory.

Thanks, I got it.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.