# Libbeat.kafka.published\_but\_not\_acked\_events

**URL:** <https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446>\
**Category:** Beats\
**Created:** [November 9, 2016, 7:50am UTC](https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446 "2016-11-09T07:50:35Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aditya\_Srivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_srivastava/32/43287_2.png) [@Aditya\_Srivastava](https://discuss.elastic.co/u/Aditya_Srivastava)\
**Post date:** [November 9, 2016, 7:50am UTC](https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446/1 "2016-11-09T07:50:35Z")

</div>

my filebeat config file:  
filebeat:  
prospectors:  
-  
paths:  
- /var/log/xyz.log  
document\_type: log  
fields: {log\_type: log1}  
-  
paths:  
- /var/log/abc.log  
document\_type: log  
fields: {log\_type: log2}  
output:  
kafka:  
hosts: ["localhost:9092"]  
topic: "%{[log\_type]}"

Now I am trying to create two different kafka topics for 2 separate files but unable to do so.  
My filebeat is running but keeps on printing the below lines-:

INFO Non-zero metrics in the last 30s: libbeat.kafka.published\_but\_not\_acked\_events=890384 libbeat.kafka.call\_count.PublishEvents=222596

Any help on this would be appreciated.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [November 9, 2016, 10:49am UTC](https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446/2 "2016-11-09T10:49:51Z")

</div>

Could you have a look at the log file? There should be quite a few error messages. Please share this part.

---

<div class="post-metadata">

**Author:** ![Aditya\_Srivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_srivastava/32/43287_2.png) [@Aditya\_Srivastava](https://discuss.elastic.co/u/Aditya_Srivastava)\
**Post date:** [November 9, 2016, 11:08am UTC](https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446/3 "2016-11-09T11:08:49Z")

</div>

Thanks for reply. The log file is ok bçoz if i give static name for topic, data is coming to kafka topic.  
ex. topic: "test"  
This way the data is coming at kafka topic

But when i try to pass dynamic value for topic name (as given in original post), I do not recieve any data nor any kind of topic is created at kafka.

Even though if you want to have a look at log file its like  
"nov 9 16:23:45 10.0.2.9 INFO [application level message]",

Lines in same format are written into log file every time  
Just wanted to know is the configuration of filebeat correct?????

---

<div class="post-metadata">

**Author:** ![Aditya\_Srivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aditya_srivastava/32/43287_2.png) [@Aditya\_Srivastava](https://discuss.elastic.co/u/Aditya_Srivastava)\
**Post date:** [November 10, 2016, 7:45am UTC](https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446/4 "2016-11-10T07:45:29Z")

</div>

Found the solution to it. The fields defined under paths are local and therefore cannot be accessed outside that path. To make the fields global we need to add following lines after fields parameter  
fields\_under\_root: true

This makes the fields key accessible from outside the path as well.  
Hope I understood this correctly. Please let me know if I wrote anything wrong.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [November 10, 2016, 12:54pm UTC](https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446/5 "2016-11-10T12:54:19Z")

</div>

you don't need `fields_under_root`. Using `topic: "%{[fields.log_type]}"` should do the trick.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 30, 2016, 7:50am UTC](https://discuss.elastic.co/t/libbeat-kafka-published-but-not-acked-events/65446/6 "2016-11-30T07:50:39Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.