# Issue with sending logs in secure Kafka

**URL:** <https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [July 27, 2020, 8:09am UTC](https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707 "2020-07-27T08:09:32Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![svasilyev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/svasilyev/32/72675_2.png) [@svasilyev](https://discuss.elastic.co/u/svasilyev)\
**Post date:** [July 27, 2020, 8:09am UTC](https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707/1 "2020-07-27T08:09:32Z")

</div>

my problem looks the same [https://discuss.elastic.co/t/issue-with-sending-to-kafka-0-10-2-1-over-tls/111252](https://discuss.elastic.co/t/issue-with-sending-to-kafka-0-10-2-1-over-tls/111252)  
but my version work

however my logs are written for some time (different on several servers) and then they stop getting into kafka.

In the logs everything looks like the recording is in progress

> {"log":"2020-07-27T10:44:53.010+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:221\u0009retryer: send unwait signal to consumer\n","stream":"stderr","time":"2020-07-27T07:44:53.010977285Z"}  
> {"log":"2020-07-27T10:44:53.010+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:225\u0009 done\n","stream":"stderr","time":"2020-07-27T07:44:53.011051349Z"}  
> {"log":"2020-07-27T10:44:55.970+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:221\u0009retryer: send unwait signal to consumer\n","stream":"stderr","time":"2020-07-27T07:44:55.97124783Z"}  
> {"log":"2020-07-27T10:44:55.970+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:225\u0009 done\n","stream":"stderr","time":"2020-07-27T07:44:55.971288498Z"}  
> {"log":"2020-07-27T10:45:14.883+0300\u0009INFO\u0009[monitoring]\u0009log/log.go:145\u0009Non-zero metrics in the last 30s\u0009{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":448180,"time":{"ms":136}},"total":{"ticks":1085250,"time":{"ms":324},"value":1085250},"user":{"ticks":637070,"time":{"ms":188}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":11},"info":{"ephemeral\_id":"74c6f4a6-9481-4bb3-b217-e59df60c1494","uptime":{"ms":89640092}},"memstats":{"gc\_next":563911584,"memory\_alloc":291324664,"memory\_total":27734031800},"runtime":{"goroutines":92}},"filebeat":{"harvester":{"files":{"de8f4d01-ab97-49e9-9e67-405ec97e1eee":{"size":2332}},"open\_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"batches":2,"failed":2,"total":2}},"outputs":{"kafka":{"bytes\_read":198548,"bytes\_write":3822}},"pipeline":{"clients":1,"events":{"active":4117,"retry":2}}},"registrar":{"states":{"current":2}},"system":{"load":{"1":0,"15":0.05,"5":0.01,"norm":{"1":0,"15":0.025,"5":0.005}}}}}}\n","stream":"stderr","time":"2020-07-27T07:45:14.883331068Z"}  
> {"log":"2020-07-27T10:45:24.015+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:221\u0009retryer: send unwait signal to consumer\n","stream":"stderr","time":"2020-07-27T07:45:24.015564821Z"}  
> {"log":"2020-07-27T10:45:24.015+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:225\u0009 done\n","stream":"stderr","time":"2020-07-27T07:45:24.015602753Z"}  
> {"log":"2020-07-27T10:45:25.858+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:221\u0009retryer: send unwait signal to consumer\n","stream":"stderr","time":"2020-07-27T07:45:25.858744677Z"}  
> {"log":"2020-07-27T10:45:25.858+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:225\u0009 done\n","stream":"stderr","time":"2020-07-27T07:45:25.858780005Z"}  
> {"log":"2020-07-27T10:45:41.738+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:221\u0009retryer: send unwait signal to consumer\n","stream":"stderr","time":"2020-07-27T07:45:41.738324614Z"}  
> {"log":"2020-07-27T10:45:41.738+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:225\u0009 done\n","stream":"stderr","time":"2020-07-27T07:45:41.738372954Z"}  
> {"log":"2020-07-27T10:45:42.476+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:221\u0009retryer: send unwait signal to consumer\n","stream":"stderr","time":"2020-07-27T07:45:42.477260785Z"}  
> {"log":"2020-07-27T10:45:42.476+0300\u0009INFO\u0009[publisher]\u0009pipeline/retry.go:225\u0009 done\n","stream":"stderr","time":"2020-07-27T07:45:42.477311442Z"}  
> {"log":"2020-07-27T10:45:44.883+0300\u0009INFO\u0009[monitoring]\u0009log/log.go:145\u0009Non-zero metrics in the last 30s\u0009{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":448330,"time":{"ms":151}},"total":{"ticks":1085600,"time":{"ms":353},"value":1085600},"user":{"ticks":637270,"time":{"ms":202}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":9},"info":{"ephemeral\_id":"74c6f4a6-9481-4bb3-b217-e59df60c1494","uptime":{"ms":89670094}},"memstats":{"gc\_next":563911584,"memory\_alloc":301172920,"memory\_total":27743880056},"runtime":{"goroutines":86}},"filebeat":{"harvester":{"files":{"de8f4d01-ab97-49e9-9e67-405ec97e1eee":{"size":1969}},"open\_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"batches":4,"failed":6,"total":6}},"outputs":{"kafka":{"bytes\_read":210704,"bytes\_write":4056}},"pipeline":{"clients":1,"events":{"active":4117,"retry":6}}},"registrar":{"states":{"current":2}},"system":{"load":{"1":0,"15":0.05,"5":0.01,"norm":{"1":0,"15":0.025,"5":0.005}}}}}}\n","stream":"stderr","time":"2020-07-27T07:45:44.883436635Z"}  
> {"log":"2020-07-27T10:46:14.887+0300\u0009INFO\u0009[monitoring]\u0009log/log.go:145\u0009Non-zero metrics in the last 30s\u0009{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":448490,"time":{"ms":164}},"total":{"ticks":1085910,"time":{"ms":319},"value":1085910},"user":{"ticks":637420,"time":{"ms":155}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral\_id":"74c6f4a6-9481-4bb3-b217-e59df60c1494","uptime":{"ms":89700096}},"memstats":{"gc\_next":563911584,"memory\_alloc":309988400,"memory\_total":27752695536},"runtime":{"goroutines":89}},"filebeat":{"harvester":{"files":{"de8f4d01-ab97-49e9-9e67-405ec97e1eee":{"size":2695}},"open\_files":1,"running":1}},"libbeat":{"config":{"module":{"running":0}},"outputs":{"kafka":{"bytes\_read":196522,"bytes\_write":3783}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":2}},"system":{"load":{"1":0,"15":0.05,"5":0.01,"norm":{"1":0,"15":0.025,"5":0.005}}}}}}\n","stream":"stderr","time":"2020-07-27T07:46:14.888155274Z"}

my config filebeat.yml

> name: host1  
> logging.level: info  
> filebeat.inputs:
> 
> - type: log  
> enabled: true  
> paths:
> - /var/lib/docker/containers/_/_-json.log  
> fields:  
> log\_topic: 'containerlogs'
> 
> # 
> 
> output.kafka:  
> hosts: ["host1:9093", "host2:9093"]  
> username: 'filebeat'  
> password: 'filebeat-password'  
> ssl.enabled: true  
> ssl.certificate\_authorities: ["root-ca.crt"]  
> topic: 'containerlogs'  
> compression: none  
> max\_retries: -1  
> backoff.max: 10s

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [July 27, 2020, 8:47am UTC](https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707/2 "2020-07-27T08:47:24Z")

</div>

This is weird indeed. Could you check with debug logs enabled for filebeat?

---

<div class="post-metadata">

**Author:** ![svasilyev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/svasilyev/32/72675_2.png) [@svasilyev](https://discuss.elastic.co/u/svasilyev)\
**Post date:** [July 27, 2020, 9:51am UTC](https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707/3 "2020-07-27T09:51:36Z")

</div>

> [@mtojek](#):
>
> This is weird indeed. Could you check with debug logs enabled for filebeat?

> <https://gist.github.com/vasilievs/58307ee3295cc92126b566dc3d5090b1>

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [July 27, 2020, 2:37pm UTC](https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707/4 "2020-07-27T14:37:58Z")

</div>

I don't see anything worrying in this dump. Could you please use wireshark or tcpdump to verify if events are sent to kafka? You should be able to the data flowing. At the moment I'm not able to decide which part might be broken.

---

<div class="post-metadata">

**Author:** ![svasilyev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/svasilyev/32/72675_2.png) [@svasilyev](https://discuss.elastic.co/u/svasilyev)\
**Post date:** [July 28, 2020, 8:45am UTC](https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707/5 "2020-07-28T08:45:35Z")

</div>

I think the problem is with kafka.  
i set up a kafka listener without sasl and ssl (plain\_text).  
After that, the logs are added to the kafka.

But this is a problem, I need sasl + ssl.  
I'm trying to add the keep\_alive 300s option (to keep the connection open) but it doesn't work as I expect

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 25, 2020, 10:45am UTC](https://discuss.elastic.co/t/issue-with-sending-logs-in-secure-kafka/242707/6 "2020-08-25T10:45:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.