Logstash container not receiving log files from Filebeats running on host

I am running a RHEL VM with Filebeats installed and three Podman containers running Kibana, Elasticsearch, and Logstash. Almost everything works, except for sending files from Filebeats to my Logstash container's pipeline.
I have tried and ensured the following:

  1. I verified my host machine's port 5044 is listening via ss -tulpn and that it's firewall is permitting traffic on the port. I can make a successful telnet connection to it. I've also ran Filebeat test output to get:
Logstash: 10.0.2.16:5044...
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.16
    dial up... OK
  TLS... WARN secure connection disabled
  talk to server... OK
  1. The Logstash Podman container was correctly mapped to the 5044 port upon execution. Verified by:
elastic@elk-server$ podman port ls01
5044/tcp -> 0.0.0.0:5044
9600/tcp -> 0.0.0.0:9600
  1. I've tested a Filebeat config which outputs to the host machine's console and it works. I've also tested Logstash and successfully ran pipelines which ingest a log file from a directory from within the container, Logstash can output both to stdout and Elasticsearch fine.
  2. I've continuously cleared both the Filebeat /var/lib/filebeat/registry/filebeat registry directory as well as my data/sincebd_ directory in Logstash
  3. I've also tried replacing my host machines IP (10.0.2.16) with localhost and 127.0.0.1 in my filebeat.yml file (shouldn't make a difference)
  4. I've put SELinux in permissive mode
  5. I don't have any TLS security configured between Filebeats and Logstash.

The contents of my filebeat.yml is:

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

filebeat.inputs:
- type: filestream
  enabled: true
  id: FreeIPA-audit-log
  paths:
    - /home/elastic/audit.test.txt
  parsers:  
    - multiline:
        type: pattern
        pattern: '^\s*$'
        negate: true
        match: after

output.logstash:
  hosts: ["10.0.2.16:5044"]

And, the contents of my logstash.conf pipeline is:

input {
  beats {
    port => 5044
  }
}

output {
  file {
    path => "/usr/share/logstash/output.log"
    codec => line { format => "%{message}" }
  }
}

I have also attempted to run a Logstash pipeline via command-line and output to stdout instead of a file, which also didn't work. I do not know what else to try at this point. Thank you for the help.

I figured out I was missing the

enabled: true

configuration line in the output section of filebeat.yml
I had it under the input, but, was not aware it was also needed under the output.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.