I'm on latest 5.0 Elastic version of Elastic/Logstash/Kibana/Beats and successfully using Winlogbeat to send multiple Windows server security logs to Logstash which outputs to Elasticsearch which are successfully viewable in Kibana, etc.
I'm now having issues sending the security log from a domain controller using the same configuration. Winlogbeat runs, but the process immediately begins consuming between 15-20% of cpu continuously and no events are ever visible in Kibana. I've used both the logstash and elasticsearch outputs in the winlogbeat.yml but the results have been the same.
Looking at the DC's I noticed that sometime in the past the security log size has been increased from default to 5G (sigh, don't ask). Since I'm having no issues with Windows servers using default security logs, I'm suspecting this is related.
So now I'm looking for ways to first confirm if this suspicion is correct and then maybe adjust my configuration to overcome this? I've already tried:
winlogbeat.event_logs: - name: Security ignore_older: 1h
in the winlogbeat.yml
There is no indication in logstash-plain.log of connection reset (which does happen sometimes if I don't filter by ignore_older: 1h) and the local winlogbeat log file reports a continuous stream of:
> 2016-11-03T08:56:40-05:00 INFO EventLog[Security] Successfully published 100 events > 2016-11-03T08:56:41-05:00 INFO EventLog[Security] Successfully published 100 events > 2016-11-03T08:56:41-05:00 INFO Non-zero metrics in the last 30s: libbeat.publisher.published_events=4700 published_events.total=4700 libbeat.logstash.published_and_acked_events=4700 msg_file_cache.SecurityHits=4700 libbeat.logstash.publish.write_bytes=1191046 published_events.Security=4700 libbeat.logstash.call_count.PublishEvents=47 libbeat.logstash.publish.read_bytes=282 > 2016-11-03T08:56:41-05:00 INFO EventLog[Security] Successfully published 100 events > 2016-11-03T08:56:42-05:00 INFO EventLog[Security] Successfully published 100 events
Any suggestions for how to troubleshoot/proceed?