# Filebeat + Kafka error

**URL:** <https://discuss.elastic.co/t/filebeat-kafka-error/109361>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 28, 2017, 10:24am UTC](https://discuss.elastic.co/t/filebeat-kafka-error/109361 "2017-11-28T10:24:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![kamandohl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kamandohl/32/25391_2.png) [@kamandohl](https://discuss.elastic.co/u/kamandohl)\
**Post date:** [November 28, 2017, 10:24am UTC](https://discuss.elastic.co/t/filebeat-kafka-error/109361/1 "2017-11-28T10:24:27Z")

</div>

I have Googled enough, but unable to find the error I am currently seeing.

Filebeat - 5.6.4 and 6.0.0  
Kafka - 10.1

> 2017/11/27 13:10:03.602226 log.go:16: WARN kafka message: Initializing new client  
> 2017/11/27 13:10:03.602342 log.go:12: WARN client/metadata fetching metadata for all topics from broker localhost  
> 2017/11/27 13:10:03.603931 log.go:12: WARN Connected to broker at localhost (unregistered)  
> 2017/11/27 13:10:03.605022 log.go:16: WARN kafka message: client/metadata got error from broker while fetching metadata:%!(EXTRA \*errors.errorString=unexpected EOF)  
> 2017/11/27 13:10:03.605074 log.go:12: WARN Closed connection to broker localhost  
> 2017/11/27 13:10:03.611668 log.go:12: WARN client/brokers resurrecting 3 dead seed brokers  
> 2017/11/27 13:10:03.611682 log.go:12: WARN client/metadata retrying after 250ms... (3 attempts remaining)

`filebeat.yml`

> filebeat.prospectors:
> 
> type: log  
> enabled: true  
> - /tmp/test.log
> 
> output.kafka:  
> enabled: true  
> hosts: ["localhost1:9093","localhost2","localhost3:9093"]  
> topic: '%{[type]}'  
> ssl:  
> ssl.certificate\_authorities: "ca.pem"  
> ssl.certificate: : "cert.pem"  
> ssl.key: "key.pem"  
> key\_passphrase: \*\*\*\*\*

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [November 28, 2017, 2:39pm UTC](https://discuss.elastic.co/t/filebeat-kafka-error/109361/2 "2017-11-28T14:39:50Z")

</div>

Please format logs and configs using the `</>`-button to make your post more readable (configs are sensitive to indentation, without formatting it's hard to check configs being correct).

1. The broker is closing the connection. Have you check the kafka logs?

2. The SSL/TLS settings must be configured via `ssl` namespace, not `tls`. Looks like you the beat is trying plain text connection, while kafka is expecting a SSL/TLS handshake.

3. remove duplicates from `hosts` setting. As you only have one broker, remove the others. The `hosts` setting is required for cluster connection bootstrapping only (get metadata to find all nodes in a cluster).

4. From you configs it looks like you want to have client authentication. But I'm missing the CA configuration for accepting the servers certificates.

Can you try without TLS first? Then add TLS server authentication and finally enable client authentication? In general properly setting up (or worse debugging) TLS can be somewhat difficult at times, better do it step by step.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 26, 2017, 2:40pm UTC](https://discuss.elastic.co/t/filebeat-kafka-error/109361/3 "2017-12-26T14:40:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.