Three types of WARN/ERRORs in filebeat logs

Let's say, it depends.

Each partition has an active leader. If the leader of a partition is not reachable, no data can be published to the affected partitions. A broker can be leader of multiple partitions, but a partition can have only one leader. Connecting to a different broker gives you no guarantees the partitions leader is reachable.

With kafka it is the client choosing which partitions data should be send to. It's up to you to decide if and how to distribute data. It's also up to you to decide if the client should pause publishing if any partition (broker being the leader) is not reachable.

The risk with not pausing is, all events might eventually end up in only one partition (worst case scenario), potentially killing the kafka cluster due to one broker running out of disk space. When sizing your cluster and configuring retention policies you have to take disk usage into account (default retention policy is by time).

In the meantime the client libs used by beats try to reconnect to failed brokers + update meta-data every so often.

If you are constantly facing network/connection issues, you should have a look at your network infrastructure. Kafka provides load-balancing, queuing and some resiliency/HA, but if you have often or long standing imbalances when publishing events you are jeopardising overall systems stability and throughput. e.g. disk usage not in balance between the different brokers/nodes. Plus each consumer in a consumer group exclusively reads from at least one partition (number of partitions gives you max parallelisation support among consumer group, having N partitions only up to N consumers in a consumer group can be served). Imbalances in data distribution enforced by producers can lead to imbalances in parallelised downstream systems/processing.

Check kafka output partitioning settings: https://www.elastic.co/guide/en/beats/filebeat/current/kafka-output.html#_partition

The default is:

kafka.output:
  partition.round_robin:
    group_events: 1
    reachable_only: false // set to true to not pause publishing if one parition becomes unavailable

Most of the functionality is provided by the kafka client libs used by beats. Client behavior is not special in beats, but common among all projects using kafka.