# Filebeat do not send logs to kafka

**URL:** <https://discuss.elastic.co/t/filebeat-do-not-send-logs-to-kafka/238718>\
**Category:** Beats\
**Created:** [June 25, 2020, 4:00pm UTC](https://discuss.elastic.co/t/filebeat-do-not-send-logs-to-kafka/238718 "2020-06-25T16:00:07Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ramgold9](https://avatars.discourse-cdn.com/v4/letter/r/57b2e6/32.png) [@ramgold9](https://discuss.elastic.co/u/ramgold9)\
**Post date:** [June 25, 2020, 4:00pm UTC](https://discuss.elastic.co/t/filebeat-do-not-send-logs-to-kafka/238718/1 "2020-06-25T16:00:08Z")

</div>

Hello Everyone,

We are trying implement a centralized logging solution filebeats (on eks) -\>kafka -\> elasticsearch(aws). I have tried different versions of filebeat 6.X and 7.X with different valid versions of kafka. File beat does not run because 'Readiness probe failed: kafka output doesn't support testing' I am using helm charts to install filebeat. When exec in to filebeat pod. the module list is empty. I added. Even than the  
'Readiness probe fails and kafka is not enabled.

My question is what are module related config need to done under filebeat.yml:  
in values.yml that s used by helm. Are not all modules by default loaded ( when installed via helm)?

Thank you for the help  
Ram

```auto
      23 filebeat.config.modules:
      24 path: ${path.config}/modules.d/*.yml
      25 filebeat.modules:
     26 - module: nginx
     27 - module: kafka 

```

Below is filebeatConfig: section of values.yml

````auto
4 filebeatConfig:
  5 filebeat.yml: |
  6 filebeat.inputs:
  7 - type: log
  8 paths:
  9 - /var/log/containers/*.log
 10 processors:
 11 - add_kubernetes_metadata:
 12 host: ${NODE_NAME}
 13 matchers:
 14 - logs_path:
 15 logs_path: "/var/log/containers/"
 16 output.kafka:
 17 enabled: true
 18 hosts: ["host1:9092","host2:9092"]
 19 topic: "Application_logs"
 20 codec.format:
 21 string: '%{[@timestamp]} %{[message]}'
 22
 23 filebeat.config.modules:
 24 path: ${path.config}/modules.d/*.yml
 25 filebeat.modules:
 26 - module: nginx
 27 - module: kafka ```
````

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [June 26, 2020, 8:58am UTC](https://discuss.elastic.co/t/filebeat-do-not-send-logs-to-kafka/238718/2 "2020-06-26T08:58:45Z")

</div>

Did you verify the connectivity? Are your hosts reachable from this host?

---

<div class="post-metadata">

**Author:** ![ramgold9](https://avatars.discourse-cdn.com/v4/letter/r/57b2e6/32.png) [@ramgold9](https://discuss.elastic.co/u/ramgold9)\
**Post date:** [June 26, 2020, 9:45pm UTC](https://discuss.elastic.co/t/filebeat-do-not-send-logs-to-kafka/238718/3 "2020-06-26T21:45:33Z")

</div>

Yes, connectivity is good. I actually tested with kafka cli, to send topic an message to kafka cluster. it works

---

<div class="post-metadata">

**Author:** ![ramgold9](https://avatars.discourse-cdn.com/v4/letter/r/57b2e6/32.png) [@ramgold9](https://discuss.elastic.co/u/ramgold9)\
**Post date:** [June 26, 2020, 9:47pm UTC](https://discuss.elastic.co/t/filebeat-do-not-send-logs-to-kafka/238718/4 "2020-06-26T21:47:01Z")

</div>

Can anyone assure me that above output configuration is correct ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 24, 2020, 11:47pm UTC](https://discuss.elastic.co/t/filebeat-do-not-send-logs-to-kafka/238718/5 "2020-07-24T23:47:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.