Exploring beats for Windows EventLog capture

Elastic Stack Beats
1

My organization is working on an ELK implementation project. One area we've experienced difficulties is gathering Windows EventLog data. In the community, the current best option appears to be using a local syslog app like nxlog to forward EventLog to Logstash.

Our own current solution is utilizing custom code written in C utilizing a few open source packages to execute remote WMI queries against a list of Windows hosts to pull this data back. It periodically runs (every 5 minutes currently). We then send it to Logstash. We like this approach as it doesn't require a heavy client/service to be deployed and configured on each Windows host. We simply maintain a list of Windows hosts. Of course, it has a bit of latency built-in (5 minute run interval). This process currently runs on Ubuntu 14 (yes, a Linux box is acting as our Windows EventLog collector).

I'm curious if this sounds like a candidate for beats implementation. I haven't explored what it would take to normalize and open-source our code, or adapt it for beats use but I'd love to talk to someone if there's interest.

We originally prototyped with just 25 hosts but have now scaled up to 250-300 and once our ELK stack is a bit more mature (currently demo/prototype) we'll be running this against 1,500-1,800 Windows hosts.

2

Hi John! We plan to re-implement Logstash Forwarder on top of the Beats framework (we'll call it Filebeat), and as part of that effort we're considering adding native Windows Event Log shipping functionality to this beat. We could also introduce Windows Event Log as a separate beat, but for now we're thinking that it makes sense to combine it with general file shipping capabilities. Thoughts?

Here is the issue on Filebeat:

And here are some of the LSF issues we plan on moving over, including Windows Event Log shipping:


1 Like
3

Hi Tanya, thanks for the quick reply! That's great there's a possibility EventLog processing will be part of Filebeat.

Thinking about it more, I think our process would likely be better served as an input plugin for Logstash itself, similar to how maybe the JDBC plugin works (periodic poll vs open port like other inputs). I really like not requiring an agent, although I think once Filebeat is ready for deployment we'd take another look at the problem.

I eagerly await Filebeats' release! :smile: