Filebeat could not send to the kafka

Elastic Stack Beats
1

Hello everyone, I ran into a problem that filebeat does not send data to kafka. Filebeat connects exactly to the kafka as it creates an empty topic, but the data does not come, I see no errors in the debug. Could there be a problem that I am connecting via haproxy?

filebeat version 7.16.2 (amd64), libbeat 7.16.2 [3c518f4d17a15dc85bdd68a5a03d5af51d9edd8e built 2021-12-18 21:04:19 +0000 UTC]

filebeat.yml

filebeat.inputs:

#- type: log
#  enabled: true
#  paths:
#      - "/var/log/nginx/skks-body.log"

filebeat.inputs:
  - type: stdin


output.kafka:
  hosts:
      - "192.168.1.35:9093"
  username: "admin"
  password: "pass"
  topic: 'test'
  partition.round_robin:
    reachable_only: false

  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000

filebeat.config.modules.path: modules.d/*.yml

haproxy conf:
изображение

Autocreated topic:

Screenshot_20220105_141330
Screenshot_20220105_1413301376×49 3.38 KB

filebeat -e -d "*"

изображение
изображение1920×783 290 KB

2

sorry, in kafka need to set advertised.listeners

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.