# Filebeat connect with Kafka Kerberos(SASL\_SSL) not working **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [August 24, 2020, 5:22pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160 "2020-08-24T17:22:40Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![chandramouli\_sriniva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandramouli_sriniva/32/74051_2.png) [@chandramouli\_sriniva](https://discuss.elastic.co/u/chandramouli_sriniva)\ **Post date:** [August 24, 2020, 5:22pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/1 "2020-08-24T17:22:40Z")
Hi - I am using Filebeat 7.9 version. using filebeat, read data from log file and push to kafka topic. our Kafka use SASL\_SSL with Kerberos mechanism. when I set up with below parms, it errors out. ```auto kerberos.enabled: "true" kerberos.auth_type: keytab kerberos.config_path: krb5.conf kerberos.realm: "" kerberos.service_name: "kafka" kerberos.username: "kafka" kerberos.keytab: keytab path ssl.enabled: "true" ssl.certificate_authorities: cer file ssl.certificate: pem file ssl.key: key file ``` Below is the error we get. Any help appreciated. Thanks. DEBUG [harvester] log/log.go:107 End of file reached: E:\Logs\file.log; Backoff now. DEBUG [kafka] kafka/client.go:277 finished kafka batch DEBUG [kafka] kafka/client.go:291 Kafka publish failed with: kafka: client has run out of available brokers to talk to (Is your cluster reachable?) Kafka publish failed with: circuit breaker is open ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 24, 2020, 5:33pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/2 "2020-08-24T17:33:47Z")
Could you please share the debug logs of Kafka? ---
**Author:** ![chandramouli\_sriniva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandramouli_sriniva/32/74051_2.png) [@chandramouli\_sriniva](https://discuss.elastic.co/u/chandramouli_sriniva)\ **Post date:** [August 24, 2020, 5:39pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/3 "2020-08-24T17:39:34Z")
I dont see any logs in broker end. I dont see parameter to provide JAAS conf file. I just give SSL and KRB5 conf here. Am I missing something from output.kafka parameters. ---
**Author:** ![chandramouli\_sriniva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandramouli_sriniva/32/74051_2.png) [@chandramouli\_sriniva](https://discuss.elastic.co/u/chandramouli_sriniva)\ **Post date:** [August 25, 2020, 5:37am UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/4 "2020-08-25T05:37:53Z")
I am able to connect logstash to kafka(with kerberos). I have given JAAS conf and krb5 conf along with SSL truststore and keystore. For Filebeat, I get "client has run out of available brokers to talk to" and "circuit breaker is open" error. I assume, between filebeat and logstash, logstash might consume more memory than filebeat and so we dont want to go with logstash. Processing wise we just read logs and send to kafka. we dont do any aggregation or processing. ---
**Author:** ![justin-jzx](https://avatars.discourse-cdn.com/v4/letter/j/c5a1d2/32.png) [@justin-jzx](https://discuss.elastic.co/u/justin-jzx)\ **Post date:** [September 1, 2020, 8:19am UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/5 "2020-09-01T08:19:35Z")
Hi, I am using filebeat 7.7.1 with kafka 0.10.0.0, and meet with the same problem. This occurs when I use the kerberos configuration. Filebeat is logging as below infinitely: > DEBUG [kafka] kafka/client.go:276 finished kafka batch > DEBUG [kafka] kafka/client.go:290 Kafka publish failed with: circuit breaker is open > INFO [publisher] pipeline/retry.go:196 retryer: send unwait-signal to consumer > INFO [publisher] pipeline/retry.go:198 done > INFO [publisher] pipeline/retry.go:173 retryer: send wait signal to consumer > INFO [publisher] pipeline/retry.go:175 done ---
**Author:** ![justin-jzx](https://avatars.discourse-cdn.com/v4/letter/j/c5a1d2/32.png) [@justin-jzx](https://discuss.elastic.co/u/justin-jzx)\ **Post date:** [September 1, 2020, 9:02am UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/6 "2020-09-01T09:02:42Z")
Is kerberos authentication based on x-pack? I use filebeat-oss-7.7.1. ---
**Author:** ![ascherbakov686](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@ascherbakov686](https://discuss.elastic.co/u/ascherbakov686)\ **Post date:** [September 28, 2020, 5:49pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/7 "2020-09-28T17:49:40Z")
Hi, i'am use filebeat 7.9.2 and have some issue with kerberos. > 2020-09-28T08:42:13.039Z DEBUG [kafka] kafka/client.go:291 Kafka publish failed with: circuit breaker is open > 2020-09-28T08:42:13.039Z DEBUG [kafka] kafka/client.go:277 finished kafka batch > 2020-09-28T08:42:13.039Z DEBUG [kafka] kafka/client.go:291 Kafka publish failed with: circuit breaker is open > 2020-09-28T08:42:13.039Z INFO [publisher] pipeline/retry.go:213 retryer: send wait signal to consumer > 2020-09-28T08:42:13.039Z INFO [publisher] pipeline/retry.go:217 done My config: ```auto - type: container enabled: true paths: - /var/log/containers/*project-prod*.log processors: - add_kubernetes_metadata: host: ${NODE_NAME} matchers: - logs_path: logs_path: "/var/log/containers/" logging: level: info selectors: ['*'] to_files: false to_syslog: false filebeat.modules: - module: kafka output.kafka: enabled: true hosts: ["kafka1.domain.ru:9092","kafka2.domain.ru:9092","kafka3.domain.ru:9092"] topic: "grace-log" #partition.round_robin: # reachable_only: false #required_acks: 1 #compression: gzip kerberos.enabled: true kerberos.auth_type: keytab kerberos.username: "svc_user" kerberos.keytab: "/tmp/keytab" kerberos.service_name: "kafka" kerberos.config_path: "/tmp/krb5.conf" kerberos.realm: "DOMAIN.RU" ``` Is there a solution to this problem ? Thank you. ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [October 15, 2020, 1:27pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/8 "2020-10-15T13:27:20Z")
It is also part of the oss version. ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [October 15, 2020, 1:27pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/9 "2020-10-15T13:27:48Z")
Please also share the debug logs of Kafka. Without looking at those, I cannot tell what is going wrong exactly. Also, which version are you using? ---
**Author:** ![ascherbakov686](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@ascherbakov686](https://discuss.elastic.co/u/ascherbakov686)\ **Post date:** [October 15, 2020, 1:41pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/10 "2020-10-15T13:41:49Z")
Hi, problem was on server side. More not actual. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 18, 2020, 6:33pm UTC](https://discuss.elastic.co/t/filebeat-connect-with-kafka-kerberos-sasl-ssl-not-working/246160/12 "2020-11-18T18:33:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.