# Filebeat. "from kafka to kafka transfer"

**URL:** <https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 25, 2022, 12:10pm UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342 "2022-01-25T12:10:51Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![faringot](https://avatars.discourse-cdn.com/v4/letter/f/e47774/32.png) [@faringot](https://discuss.elastic.co/u/faringot)\
**Post date:** [January 25, 2022, 12:10pm UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/1 "2022-01-25T12:10:51Z")

</div>

Good day,  
I'm trying to set up data transfer from one Kafka to another using Filebeat. The current version does not work with SASL\_SSL and I have downloaded 8.0.0-rc1.  
Found a new error: **"Exiting: module kafka is configured but has no enabled filesets"**

Didn't find "filesets" in documentation for version 8.0.0

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [January 25, 2022, 12:46pm UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/2 "2022-01-25T12:46:10Z")

</div>

What does ur config look like? Are u using the Kafka module or a Kafka input? It sounds like u want a Kafka input and output.

---

<div class="post-metadata">

**Author:** ![faringot](https://avatars.discourse-cdn.com/v4/letter/f/e47774/32.png) [@faringot](https://discuss.elastic.co/u/faringot)\
**Post date:** [January 25, 2022, 1:54pm UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/3 "2022-01-25T13:54:07Z")

</div>

Yes, i want kafka input and output.  
My config:

```auto
filebeat.config.modules:
  enabled: true
  path: ${path.config}/modules.d/*.yml

filebeat.inputs:
- type: kafka
  hosts:
    - bootstrap-1:9092
  topics: ["test.topic"]
  group_id: "filebeat"
  ssl.enabled: yes
  ssl.certificate_authorities: "/pathTOfile/Root.crt"
  ssl.certificate: "/pathTOfile/Cert.crt"
  ssl.key: "/pathTOfile/Key.rsa.key"
  ssl.key_passphrase: "passphrase"
  kerberos.auth_type: keytab
  kerberos.config_path: "/pathTOfile/krb5.conf"
  kerberos.keytab: "/pathTOfile/mykeytab.keytab"
  kerberos.username: "username"
  kerberos.service_name: "kafka"
  kerberos.realm: "REALM.REALM"

filebeat.modules:
- module: kafka

processors:

output.kafka:
  hosts: ["bootstrap-1:2892"]
  topic: 'test.topic'
  partition.round_robin:
    reachable_only: true
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
  ssl.enabled: yes
  ssl.certificate_authorities: ["/pathTOfile/Root.crt"]
  ssl.certificate: "/pathTOfile/Cert.crt"
  ssl.key: "/pathTOfile/Key.rsa.key"
  ssl.key_passphrase: "passphrase"
  kerberos.auth_type: keytab
  kerberos.config_path: "/pathTOfile/krb5.conf"
  kerberos.keytab: "/pathTOfile/mykeytab.keytab"
  kerberos.username: "username"
  kerberos.service_name: "kafka"
  kerberos.realm: "REALM.REALM"

```

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [January 25, 2022, 10:51pm UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/4 "2022-01-25T22:51:40Z")

</div>

Ya u don't need the Kafka module. Just the input/output.

---

<div class="post-metadata">

**Author:** ![faringot](https://avatars.discourse-cdn.com/v4/letter/f/e47774/32.png) [@faringot](https://discuss.elastic.co/u/faringot)\
**Post date:** [January 26, 2022, 7:01am UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/5 "2022-01-26T07:01:55Z")

</div>

Thx, working, but got new error:

```auto
{"file.name":"kafka/input.go","file.line":152},"message":"Error initializing kafka consumer group"
and
{"message":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)"}

```

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [January 26, 2022, 9:46pm UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/6 "2022-01-26T21:46:53Z")

</div>

Are the ports for Kafka supposed to be different?

---

<div class="post-metadata">

**Author:** ![faringot](https://avatars.discourse-cdn.com/v4/letter/f/e47774/32.png) [@faringot](https://discuss.elastic.co/u/faringot)\
**Post date:** [January 27, 2022, 5:59am UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/7 "2022-01-27T05:59:38Z")

</div>

Yes, it's two instance of kafka on 1 server. All working, check it

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [January 27, 2022, 7:21pm UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/8 "2022-01-27T19:21:38Z")

</div>

It's hard to say. It seems that it's having issues connecting to ur Kafka instance. I'd check all the connection parameters.

---

<div class="post-metadata">

**Author:** ![faringot](https://avatars.discourse-cdn.com/v4/letter/f/e47774/32.png) [@faringot](https://discuss.elastic.co/u/faringot)\
**Post date:** [January 28, 2022, 7:31am UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/9 "2022-01-28T07:31:26Z")

</div>

This connection parameters use in another apps like console-consumer, KafkaHQ, CMAK etc.  
Not working only in filebeat ;(

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 25, 2022, 9:31am UTC](https://discuss.elastic.co/t/filebeat-from-kafka-to-kafka-transfer/295342/10 "2022-02-25T09:31:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
