Filebeat with 2 kafka outputs

Elastic Stack Beats
1

Hello !

I have an issue with Filebeat when I try to send data logs to 2 Kafka nodes at the same time

The following is the Output Kafka section of the filebeat.yml file:
output.kafka:
enabled: true
hosts: [ "192.168.xxx.xx:9092", "192.168.zzz.zz:9092" ]
topic: "syslog"
timeout: 30s
max_message_bytes: 1000000

Both kafka services are running, but only the second node get data. I mean Only the Kafka node 192.168.zzz.zz get the data Filebeat have sent it.

If I exchange the IP adressess, occurs that the second IP address get the data log.

Why occurs that ? What other configurations are need to implement this use case? I need send the data to both kafka outputs.

Thanks in advance

Regards

hernan d

2

As far as I know Filebeat can only send to a single destination. If you specify multiple destination hosts I would therefore expect it to use this for failover whenever necessary. If you send the data through Logstash you should be able to write to multiple outputs from there.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.