# Unable to receive the log from filebeat to kafka

**URL:** https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224
**Category:** Beats
**Tags:** filebeat
**Created:** [December 26, 2017, 2:51pm UTC](https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224 "2017-12-26T14:51:41Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![saravana\_ganesh](https://avatars.discourse-cdn.com/v4/letter/s/59ef9b/32.png) [@saravana\_ganesh](https://discuss.elastic.co/u/saravana_ganesh)
#### Post date: [December 26, 2017, 2:51pm UTC](https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224/1 "2017-12-26T14:51:41Z")

</div>

Hi,

We are using kafka 2.12 and configure the output as below in the filebeat.

filebeat.prospectors:

- type: log

#============================= Filebeat modules ===============================

filebeat.config.modules:

# Glob pattern for configuration loading

path: ${path.config}/modules.d/\*.yml

# Set to true to enable config reloading

reload.enabled: false

#==================== Elasticsearch template setting ==========================

setup.template.settings:  
index.number\_of\_shards: 3  
#index.codec: best\_compression  
#\_source.enabled: false

#-------------------------- Kafka Output ----------------------------------  
output.kafka:

# initial brokers for reading cluster metadata

hosts: ["localhost:9092"]

# message topic selection + partitioning

topic: 'logs'  
partition.round\_robin:  
reachable\_only: false

required\_acks: 1  
compression: gzip  
max\_message\_bytes: 1000000

logging:  
files:  
rotateeverybytes: 10485760 # = 10MB

We have check the logs in the kafka there is no log on it.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [December 28, 2017, 2:13am UTC](https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224/2 "2017-12-28T02:13:10Z")

</div>

Could you share the Filebeat version you are using and the filebeat log file? Please make sure to format it (same for the config above).

---

<div class="post-metadata">

### Author: ![saravana\_ganesh](https://avatars.discourse-cdn.com/v4/letter/s/59ef9b/32.png) [@saravana\_ganesh](https://discuss.elastic.co/u/saravana_ganesh)
#### Post date: [December 28, 2017, 5:58am UTC](https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224/3 "2017-12-28T05:58:11Z")

</div>

filebeat-6.1.1

Filebeat log given below :

INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30000 beat.memstats.gc\_next=4473924 beat.memstats.memory\_alloc=3313200 beat.memstats.memory\_total=3313200 filebeat.harvester.open\_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.pipeline.clients=0 libbeat.pipeline.events.active=0 registrar.states.current=0

---

<div class="post-metadata">

### Author: ![ktella](https://avatars.discourse-cdn.com/v4/letter/k/71e660/32.png) [@ktella](https://discuss.elastic.co/u/ktella)
#### Post date: [December 29, 2017, 1:35am UTC](https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224/4 "2017-12-29T01:35:15Z")

</div>

Not trying to hijack the thread, but i too have the exact issue.  
Created a new thread "Unable send logs from filebeat 6 to kafka"  
I have tried with my old filebeat 5.x version and its working perfect.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [January 3, 2018, 10:58pm UTC](https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224/5 "2018-01-03T22:58:49Z")

</div>

Can you both share your full log (best in debug mode) and config?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 31, 2018, 10:58pm UTC](https://discuss.elastic.co/t/unable-to-receive-the-log-from-filebeat-to-kafka/113224/6 "2018-01-31T22:58:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
