Filebeat in Docker can't output to Kafka

I'm trying to use filebeat in Docker to read log files from local machine and output to Kafka. But it doesn't work properly, doesn't output any message to Kafka. And I tested Kafka on my local machine, it works well itself.

On my local machine, the file and folder structure like this:

The content of Dockerfile like below:

COPY filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
RUN chown root:filebeat /usr/share/filebeat/filebeat.yml
USER filebeat

The Content of filebeat.yml like below:

#=========================== Filebeat inputs =============================


- type: log
  enabled: true
  # Paths that should be crawled and fetched. Glob based paths.
    - /Users/lachen/Documents/Log-Monitor/nifi-bootstrap-3.log

#----------------------------- Kafka output --------------------------------
 # is the private ip address of my local machine
 #  even though I use localhost as ["localhost:9092"], and it doesn't work properly
  hosts: [""]
  topic: pam-filebeat-test2
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000

1. I build the filebeat image using below command:
docker image build -t filebeat:1.0.2 .

2. I run the filebeat container using below command
docker container run -u 0 --name filebeat-container filebeat:1.0.2

I also tried to run the container using command like below, it doesn't work as well:
docker container run -u 0 -e strict.perms=false --name filebeat-container filebeat:1.0.2

3. I check the topic in kafka using below command:
kafka-console-consumer --bootstrap-server localhost:9092 --topic pam-filebeat-test2 --from-beginning

4. add new contents to the logfiles

echo "1">>nifi-bootstrap-3.log
echo "2">>nifi-bootstrap-3.log
echo "3">>nifi-bootstrap-3.log

5. go back and check the kafka, it didn't receive any message from the filebeat.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.