Filebeat with event-hub-kafka output, pulish fails: client has run out of available brokers to talk to

Elastic Stack Beats
1

Description

I am using Filebeat to stream my log file to azure event hub and with the configuration as kafka output, the connection to host ip can be established (read from the log below), but when it try to publish topics there is error "Kafka publish failed with: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)". I am sure the setting of connection string and evenhub names, topics are correct cause I test with python script and event hub can receive the messages. When it come to Filebeat, only see requests but no message. Is it due to authentication issues? And my computer need proxy seeting, while there is no "proxy_url" variable avaliable in output.kafka. I'm using filebeat-7.12.0-windows-x86_64.

How to reproduce

The filebeat.yml file looks like:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\DataPipeline\DataOut\*.ndjson
  json.key_under_root: true
  json.add_error_key: true
setup.template.settings:
  index.number_of_shards: 1
output.kafka:
  topic: "ehpwt"
  required_acks: 1
  client_id: filebeat
  version: '1.0.0'
  hosts:
    - "ehnamespace.servicebus.windows.net:9093"
  username: "$ConnectionString"
  password: "(The connection string - secure)"
  ssl.enabled: true
  compression: none
  max_message_bytes: 1000000
  partition.round_robin:
    reachable_only: true
logging.level: debug
logging.to_files: true
logging.files:
  path: C:\log\filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

and my error log shows:
2021-06-16T21:21:26.870+0200 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to kafka(ehnspwt.servicebus.windows.net:9093)
2021-06-16T21:21:26.882+0200 DEBUG [kafka] kafka/client.go:100 connect: [ehnspwt.servicebus.windows.net:9093]
2021-06-16T21:21:26.886+0200 INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to kafka(ehnspwt.servicebus.windows.net:9093) established
2021-06-16T21:21:26.891+0200 DEBUG [harvester] log/log.go:107 End of file reached: C:\DataPipeline\DataOut\test01 - Copy (4).ndjson; Backoff now.
2021-06-16T21:21:27.692+0200 DEBUG [kafka] kafka/client.go:371 finished kafka batch
2021-06-16T21:21:27.693+0200 DEBUG [kafka] kafka/client.go:385 Kafka publish failed with: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
2021-06-16T21:21:27.693+0200 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer
2021-06-16T21:21:27.701+0200 INFO [publisher] pipeline/retry.go:223 done
2021-06-16T21:21:28.496+0200 DEBUG [kafka] kafka/client.go:371 finished kafka batch
2021-06-16T21:21:28.496+0200 DEBUG [kafka] kafka/client.go:385 Kafka publish failed with: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

Update

An update is that: I tried same setting with filebeat-7.0.0-beta1-windows-x86_64 which shows totally different logs,

2021-06-17T14:08:03.188+0200 INFO kafka/log.go:53 Failed to connect to broker ehnspwt.servicebus.windows.net:9093: dial tcp: lookup ehnspwt.servicebus.windows.net: no such host

I tried the same setting with another laptop which can connect directly to internet without proxy and it works, so it's still a proxy problem. Where can I set the similar setting as "proxy_url: 'http://127.0.0.1:3128" in Filebeat with kafka output?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.