- Filebeat version: 5.0.0-alpha4
- Operating System: debian 8
I use filebeat get log from computer A, and then send to a kafka cluster group by two kafka run at two computers named B and C.
Today, I test the high avaliability of this system. below are what I do:
setting the firewall to cut off the network between A and B, and I hope the filebeat can send log to C, but not.these are the log of filebeat:
2016-07-22T15:43:34+08:00 WARN kafka message: client/metadata got error from broker while fetching metadata:%!(EXTRA *net.OpError=read tcp A's ip:37157-> B's ip:9093: i/o timeout ) 2016-07-22T15:43:34+08:00 WARN Closed connection to broker B's ip:9093 2016-07-22T15:43:34+08:00 WARN client/metadata fetching metadata for [t_log] from broker C's ip:9093 2016-07-22T15:43:34+08:00 WARN Connected to broker at C's ip:9093 (unregistered) 2016-07-22T15:43:34+08:00 WARN producer/broker/2 starting up 2016-07-22T15:43:34+08:00 WARN producer/broker/2 state change to [open] on t_log/5 2016-07-22T15:43:34+08:00 WARN producer/leader/t_log/5 selected broker 2 2016-07-22T15:43:34+08:00 WARN producer/leader/t_log/5 state change to [flushing-1] 2016-07-22T15:43:34+08:00 WARN producer/leader/t_log/5 state change to [normal]
the broker 2 is kafka on computer B.
the filebeat can't get the metadata from B because of the firewall on A, and then get metadata from C. But C can connect to B , and C think B is alive. So C send the metadata to A and suggested A also push data to B. And finally data can't send to kafka cluster.
And when you kill the kafka at B, filebeat can send data to C.
I think this is a bug, so how to fix it ?