# Winlogbeat Throughput Seems Low

**URL:** <https://discuss.elastic.co/t/winlogbeat-throughput-seems-low/142966>\
**Category:** Beats\
**Tags:** winlogbeat\
**Created:** [August 3, 2018, 7:37pm UTC](https://discuss.elastic.co/t/winlogbeat-throughput-seems-low/142966 "2018-08-03T19:37:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![BrandonF](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandonf/32/34032_2.png) [@BrandonF](https://discuss.elastic.co/u/BrandonF)\
**Post date:** [August 3, 2018, 7:37pm UTC](https://discuss.elastic.co/t/winlogbeat-throughput-seems-low/142966/1 "2018-08-03T19:37:37Z")

</div>

We are running a POC with beats to see if it can replace our current connector infrastructure. We can't seem to get winlogbeat to get past 1,500-2,000 eps. Our current connectors are able to peak at about 3k...Any suggestions here would be extremely helpful!

We've played around with quite a few variations on the configs, but we can't seem to get the boost.

Our effective config:

```auto
winlogbeat.event_logs:
    - name: ForwardedEvents
setup.template.settings:
  index.number_of_shards: 3
tags: ["XXXXXXXX"]
queue:
  mem:
    events: 65536
output.kafka:
  enabled: true
  hosts: XXXXXXXX 
  topic: winlogbeat
  username: XXXXXXXX
  password: XXXXXXXX
  version: 0.11.0
  worker: 1
  max_retries: -1
  ssl.enabled: true
xpack.monitoring:
  enabled: true
  elasticsearch:
    hosts: XXXXXXXX

```

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [August 8, 2018, 6:37pm UTC](https://discuss.elastic.co/t/winlogbeat-throughput-seems-low/142966/2 "2018-08-08T18:37:53Z")

</div>

You might be able to get a small bump by setting

```auto
winlogbeat.event_logs:
    - name: ForwardedEvents
      batch_read_size: 512

```

Last time we did a change a benchmark was done on the reader. That's without any time spent sending to an output (kafka in your case) so any numbers in a actual setup will likely be lower.

We have an open issue to try another means of reading events that might make it faster. [https://github.com/elastic/beats/issues/6585](https://github.com/elastic/beats/issues/6585)

Each event log reader is independent so if you could "shard" your forwarded events across more than one event log you could have multiple readers consuming. See [Creating Custom Windows Event Forwarding Logs](https://blogs.technet.microsoft.com/russellt/2016/05/18/creating-custom-windows-event-forwarding-logs/). For example create a WEC-Powershell and WEC-System to receive events on the collector rather than dumping them all in ForwardedEvents. Then setup Winlogbeat to read from each of those logs. And see [https://github.com/palantir/windows-event-forwarding/tree/master/windows-event-channels](https://github.com/palantir/windows-event-forwarding/tree/master/windows-event-channels).

---

<div class="post-metadata">

**Author:** ![MikeKemmerer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mikekemmerer/32/34385_2.png) [@MikeKemmerer](https://discuss.elastic.co/u/MikeKemmerer)\
**Post date:** [August 10, 2018, 2:54pm UTC](https://discuss.elastic.co/t/winlogbeat-throughput-seems-low/142966/3 "2018-08-10T14:54:45Z")

</div>

Andrew,

I, too, am looking for higher throughput windows event log consumption via winlogbeat -\> kafka to replace our legacy log management system. I've made the change you recommended, and that does not make a difference. Turning on DEBUG level logging shows errors thrown and adjusting the batch read size down to 256. Other changes I've made without success:

- added a processor to drop the message field
- set kafka acks to 0
- increased the max message size on kakfa broker and winlogbeat to 10M
- increased and decreased queue sizes
- increased and decreased flush timeout
- tested various numbers of workers, and max\_procs

Even after running `winlogbeat -v -N`, I'm not seeing anywhere near the ~3k messages per second rate in the benchmark: [https://github.com/elastic/beats/pull/6150#discussion\_r163857460](https://github.com/elastic/beats/pull/6150#discussion_r163857460).

Currently, we cannot modify our WEC setup so it does not interfere with the existing (and functioning) log shipper.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 7, 2018, 2:54pm UTC](https://discuss.elastic.co/t/winlogbeat-throughput-seems-low/142966/4 "2018-09-07T14:54:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.