Filebeat with 2 kafka outputs


(Dario Rodriguez) #1

Hello !

I have an issue with Filebeat when I try to send data logs to 2 Kafka nodes at the same time

The following is the Output Kafka section of the filebeat.yml file:
output.kafka:
enabled: true
hosts: [ "192.168.xxx.xx:9092", "192.168.zzz.zz:9092" ]
topic: "syslog"
timeout: 30s
max_message_bytes: 1000000

Both kafka services are running, but only the second node get data. I mean Only the Kafka node 192.168.zzz.zz get the data Filebeat have sent it.

If I exchange the IP adressess, occurs that the second IP address get the data log.

Why occurs that ? What other configurations are need to implement this use case? I need send the data to both kafka outputs.

Thanks in advance

Regards

hernan d


(Christian Dahlqvist) #2

As far as I know Filebeat can only send to a single destination. If you specify multiple destination hosts I would therefore expect it to use this for failover whenever necessary. If you send the data through Logstash you should be able to write to multiple outputs from there.


(system) closed #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.