# Kafka output, CPU spikes if Kafka Cluster Unavailable
**URL:** https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892
**Category:** Beats
**Tags:** winlogbeat
**Created:** [May 1, 2016, 4:57pm UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892 "2016-05-01T16:57:55Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@landonix](https://discuss.elastic.co/u/landonix)
#### Post date: [May 1, 2016, 4:57pm UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/1 "2016-05-01T16:57:55Z")
I have been testing Winlogbeat by dropping event log output directly into a Kafka cluster queue. It works very well with uninterrupted/reliable network connectivity. However in my test case I wanted to see the impact if the Kafka cluster was not available and how the client device behaved.
Unfortunately, if the Kafka cluster is not available the process spikes CPU usage resulting in rapid output of the log data displayed below. The default timeouts/retries for the kafka output seemed reasonable enough that I did not specifically modify the values.
**Windows 7 Client Operating System**
**Client Config**
> ```
> winlogbeat:
> registry_file: C:/ProgramData/winlogbeat/.winlogbeat.yml
> event_logs:
> - name: Application
> ignore_older: 72h
> - name: Security
> - name: System
> output:
> kafka:
> hosts: testkafka01:9092
> topic: winlogbeat
> shipper:
> logging:
> to_files: true
> files:
> path: C:/ProgramData/winlogbeat/Logs
> rotateeverybytes: 10485760 # = 10MB
> level: info
>
> ```
**Logs**
```auto
2016-05-01T12:08:05-04:00 INFO EventLog[System] Successfully published 58 events
2016-05-01T12:08:05-04:00 INFO EventLog[Security] Successfully published 54 events
2016-05-01T12:08:05-04:00 INFO EventLog[Application] Successfully published 59 events
2016-05-01T12:08:05-04:00 INFO EventLog[System] Successfully published 8 events
2016-05-01T12:08:05-04:00 INFO EventLog[Security] Successfully published 30 events
2016-05-01T12:08:05-04:00 INFO EventLog[Application] Successfully published 52 events
2016-05-01T12:09:46-04:00 INFO EventLog[Application] Successfully published 2 events
2016-05-01T12:13:07-04:00 INFO EventLog[Application] Successfully published 2 events
2016-05-01T12:13:30-04:00 INFO EventLog[Application] Successfully published 1 events
2016-05-01T12:13:31-04:00 INFO EventLog[Security] Successfully published 2 events
2016-05-01T12:13:31-04:00 INFO EventLog[Application] Successfully published 3 events
2016-05-01T12:14:02-04:00 INFO EventLog[Application] Successfully published 2 events
2016-05-01T12:15:31-04:00 WARN producer/broker/[0 826814776496] state change to [closing] because %!s(MISSING)
2016-05-01T12:15:31-04:00 WARN Closed connection to broker [testkafka01:9092]
```
**Repeated Log Entries**
```auto
2016-05-01T12:15:31-04:00 INFO Error publishing events (retrying): write tcp 172.16.93.141:49235->172.16.93.140:9092: wsasend: An existing connection was forcibly closed by the remote host.
2016-05-01T12:15:31-04:00 INFO Error publishing events (retrying): write tcp 172.16.93.141:49235->172.16.93.140:9092: wsasend: An existing connection was forcibly closed by the remote host.
... (repeats until process killed)
```
---
### Author:  [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [May 2, 2016, 1:49pm UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/2 "2016-05-02T13:49:20Z")
There are some on going fixes being made to the Kafka output by @steffens. This is probably the same issue.
- [https://github.com/elastic/beats/issues/1432#issuecomment-215239880](https://github.com/elastic/beats/issues/1432#issuecomment-215239880)
- [https://github.com/elastic/beats/pull/1543](https://github.com/elastic/beats/pull/1543)
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [May 2, 2016, 5:03pm UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/3 "2016-05-02T17:03:56Z")
Yes, same issue. Fixed and hopefully available in alpha2 soonish.
---
### Author:  [@landonix](https://discuss.elastic.co/u/landonix)
#### Post date: [May 2, 2016, 8:45pm UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/4 "2016-05-02T20:45:27Z")
Thank you! I look forward to trying to alpha2. Any idea when soonish will be from a timing perspective?
---
### Author:  [@landonix](https://discuss.elastic.co/u/landonix)
#### Post date: [May 14, 2016, 4:38pm UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/5 "2016-05-14T16:38:44Z")
Alpha2 resolved this issue in two of our test environments.
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [May 17, 2016, 11:04am UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/6 "2016-05-17T11:04:20Z")
So is problem resolved?
---
### Author:  [@landonix](https://discuss.elastic.co/u/landonix)
#### Post date: [May 17, 2016, 11:45am UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/7 "2016-05-17T11:45:46Z")
Yes!
---
### Author:  [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [May 23, 2016, 2:44pm UTC](https://discuss.elastic.co/t/kafka-output-cpu-spikes-if-kafka-cluster-unavailable/48892/8 "2016-05-23T14:44:00Z")