# Winlogbeat performance issue **URL:** \ **Category:** Beats\ **Tags:** winlogbeat\ **Created:** [August 2, 2018, 3:02pm UTC](https://discuss.elastic.co/t/winlogbeat-performance-issue/142779 "2018-08-02T15:02:32Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Tim\_P](https://avatars.discourse-cdn.com/v4/letter/t/a8b319/32.png) [@Tim\_P](https://discuss.elastic.co/u/Tim_P)\ **Post date:** [August 2, 2018, 3:02pm UTC](https://discuss.elastic.co/t/winlogbeat-performance-issue/142779/1 "2018-08-02T15:02:32Z")
Hi, We forward all our very busy Active Directory Event logs to a single host, I then am trying to read these forwarded event logs using winlogbeat but I seem to hit a problem that it's just ignoring messages. I have no issues on the elastic side. It basically seem to be ignoring the recent updates and occasionally publishes 50 or so. I've been tailing the logs but this is not giving me any useful information about why it's not processing events. I've tried differing configs, different outputs nothing seem to resolve my issue. Windows version: Server 2016 Elasticsearch version: 6.3.2 winlogbeat version: 6.3.2 ## winlogbeat.yml winlogbeat.event\_logs: - name: ForwardedEvents ignore\_older: 4h batch\_read\_size: 1024 queue: mem: events: 32736 setup.template.name: "winlogbeat" setup.template.pattern: "ad-log-\*" fields: service: AD drop\_fields: fields: ["[host]","event\_data.ProcessName","event\_data.TransmittedServices","\_score","event\_data.LogonGuid","provider\_guid", "event\_data.KeyLength","event\_data.ProcessId","event\_data.TargetLogonGuid","source\_name", "record\_number", "thread\_id", "process\_id","event\_data.TargetUserSid","event\_data.TargetSid","event\_data.ServiceSid"] output.elasticsearch: enabled: true hosts: ["elk6-data1:9200","elk6-data2:9200","elk6-data3:9200","elk6-data4:9200","elk6-data5:9200","elk6-data6:9200"] index: "ad-log-%{+yyyy.MM.dd}" bulk\_max\_size: 0 logging.files: name: winlogbeat rotateeverybytes: 10485760 # = 10MB keepfiles: 2 xpack.monitoring.elasticsearch: Thanks Tim ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 3, 2018, 7:46am UTC](https://discuss.elastic.co/t/winlogbeat-performance-issue/142779/2 "2018-08-03T07:46:52Z")
Winlogbeat reads messages from Event Log which returns events in batches. Then the publisher does not create new smaller batches, it simply forwards batches as is. In this case `bulk_max_size` is irrelevant and does not do anything to the publisher pipeline. ---
**Author:** ![Tim\_P](https://avatars.discourse-cdn.com/v4/letter/t/a8b319/32.png) [@Tim\_P](https://discuss.elastic.co/u/Tim_P)\ **Post date:** [August 3, 2018, 11:43am UTC](https://discuss.elastic.co/t/winlogbeat-performance-issue/142779/3 "2018-08-03T11:43:27Z")
I've tried it with and without bulk\_max\_size it makes little difference, this does not explain why it's ignoring new events and yet if I run it from the command line it happily read the old events, but when it gets up to the new events it stalls. ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [August 3, 2018, 12:44pm UTC](https://discuss.elastic.co/t/winlogbeat-performance-issue/142779/4 "2018-08-03T12:44:34Z")
So the new events doesn't make it to the output? ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [August 3, 2018, 2:51pm UTC](https://discuss.elastic.co/t/winlogbeat-performance-issue/142779/5 "2018-08-03T14:51:09Z")
There are several things you can try. You can enable debug logging. Two of the more useful debug [selectors](https://github.com/elastic/beats/blob/master/winlogbeat/eventlog/eventlog.go#L35-L39) related to the reading of events logs are `eventlog` and `eventlog_detail`. Each time it asks Windows for more events [it will report](https://github.com/elastic/beats/blob/master/winlogbeat/eventlog/wineventlog.go#L194) how many it received. ```auto logging.level: debug logging.selectors: - eventlog - eventlog_detail # You might want to remove this one if it's too much output. ``` It would be helpful if you could share the Winlogbeat log output that you have. By default it logs a set of metrics every 30s that contain some information about publishing to ES as well as metrics from Winlogbeat. BTW It can actually be counter-productive to set `batch_read_size: 1024` if the messages it's receiving are large because [an error](https://github.com/elastic/beats/issues/3076) can result and it must re-read with a smaller batch size. You can monitor for this behavior by checking the metrics that are logged. There is a `read_errors` object in the metrics output that contains a mapping of error name/number (`1734` in this case) to an occurrence count. If you are hitting those errors then a batch\_read\_size of 512 might be better. Another thing you can try is to replace the Elasticsearch output with a File output and see if the blocking occurs. If so then it's probably back-presssure from ES that causes reading to stop once the queue memory fills up. Hopefully this gets you a little further in understanding and debugging your issue. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 31, 2018, 2:51pm UTC](https://discuss.elastic.co/t/winlogbeat-performance-issue/142779/6 "2018-08-31T14:51:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.