Filebeat output to Kafka GSLB

Elastic Stack Beats
1

We are trying to solve a problem here with filebeat agent, we want to introduce an GSLB infront of Kafka clusters. Basically we have 2 kafka clusters in 2 different zones, we want filebeat to output to kafka based on reachability, Incase we have Zone 1 kafka down, filebeat should be able to connect to Zone2 via same pipeline without needing any restart.

Our kafka clusters are based on RKS, and we connect it via an FQDN:port with mTLS.

I know filebeat supports output to logstash with this approach, can it also do same for output.kafka: ?

2

If you ask, can FB send data to Kafka, yes, check here. If you want a load balance, an active-passive configuration, I don't see an option to set.

3

output.kafka:
enabled: true
hosts: ["abc.domain.com:443"]
topic: filebeat1

//

abc.domain.com is DNS which resolves to an LTM beneath.

4

Can this be achieved?

5

What is exactly your issue? Is not clear.

What is working and what is not working?