Send data from filebeat to kafka(with kerberos) failed

filebeat: filebeat-7.10.2-linux-x86_64
kafka: kafka_2.13-2.6.0/

here is my filebeat config:

filebeat.inputs:
- type: log

  # Change to true to enable this input configuration.
  enabled: true
  fields_under_root: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /home/work/test/log/*.log
    #- c:\programdata\elasticsearch\logs\*
  json.keys_under_root: true
  json.add_error_key: false
  json.overwrote_keys: true
output.kafka:
  enabled: true
  hosts: ["xxx1:9093",
          "xxx2:9093"]
  topic: "test3"
  partition.round_robin:
    reachable_only: false
  compression: none
  max_message_bytes: 1000000
  sasl.mechanism:
  kerberos.enabled: true
  kerberos.auth_type: keytab
  kerberos.username: "yuenxu"
  kerberos.keytab: "/home/work/keytab/yuenxu.keytab"
  kerberos.service_name: "kafka"
  kerberos.config_path: "/home/work/krb5/krb5.conf"
  kerberos.realm: "LZ.DSCC.99.COM"

About the kafka with kerberos, I have test it with console producer/consumer. It's OK.

Here is the filebeat error message:

2021-09-17T19:31:19.916+0800	INFO	[publisher]	pipeline/retry.go:219	retryer: send unwait signal to consumer
2021-09-17T19:31:19.916+0800	INFO	[publisher]	pipeline/retry.go:223	  done
2021-09-17T19:31:19.996+0800	DEBUG	[kafka]	kafka/client.go:277	finished kafka batch
2021-09-17T19:31:19.996+0800	DEBUG	[kafka]	kafka/client.go:291	Kafka publish failed with: circuit breaker is open
2021-09-17T19:31:19.996+0800	INFO	[publisher]	pipeline/retry.go:219	retryer: send unwait signal to consumer
2021-09-17T19:31:19.996+0800	INFO	[publisher]	pipeline/retry.go:223	  done

Is there anything wrong? Thanks.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.