How can ship logs to Logstash using Windows Events?

Elastic Stack Logstash
1

Hi,

I am very new to ELK stack. I have setup a stack using two Windows Server 2012 R2 VM machines, one is for Logstash and other for Kibana and Elastic search. Initially, I was thinking to use nxlog for logs forwarding but I came to know that I may use Windows events as well using group policy.

Can anyone help me in this regard? Please refer me to step by step guide or if anyone has an experience in it, let me know.

Why I don't want to use logstash-forwarder because then I will have to install Java on all of our Servers which I don't want obviously.

Thanks.

2

Initially, I was thinking to use nxlog for logs forwarding but I came to know that I may use Windows events as well using group policy.

My Windows-fu isn't strong enough to fully comprehend what you've written, but AFAIK you can pull event logs from a remote machine, in which case you shouldn't have to be dependent on running a log shipping agent on each Windows machine. However, Logstash's eventlog input plugin doesn't seem to support this.

Why I don't want to use logstash-forwarder because then I will have to install Java on all of our Servers which I don't want obviously.

Logstash-forwarder does not have a Java dependency, but Logstash proper does. If you don't want a JVM dependency NxLog seems like a good choice.

3

Hi Magnus,

Thanks for your reply. As I am a newbie, this is a news to me that I can pull logs from Windows machine, that's awesome. Then why people use log shipping agents on their computer, I am just curious?

So, could you please refer me to few examples of conf files where users have pulled the logs out of Windows machines. I really need to see the examples of conf files.

4

As I said, I think Windows itself supports reading event log from remote machines, but Logstash doesn't support it.

5

If I want to ship different kind of logs to logstash, i.e. Windows Event and IIS. Should I mention two different IP addresses and ports in input section of logstash.conf. For example:

input {
tcp {
host => "127.0.0.1"
port => 3514
}
tcp {
type => "eventlog"
host => "10.1.1.2"
port => 3515
format => 'json'
}
}

6

Why different IP addresses? Normally you don't set the host option for inputs at all.

If you need to use different codecs or set different types you should probably listen on different ports. However, you can also change the type based on the contents of the message. So in the end it depends. What's going to send data on those TCP ports?

7

I want to send Windows Event logs and IIS from our webservers. So, how should the conf file looks like then?

8

Is this code looks good:

input {
tcp {
type => "WindowsEventLog"
port => 5544
codec => "line"
}
}

filter{
if [type] == "WindowsEventLog" {
json{
source => "message"
}
if [SourceModuleName] == "eventlog" {
mutate {
replace => [ "message", "%{Message}" ]
}
mutate {
remove_field => [ "Message" ]
}
}
}
}

output {
elasticsearch {
host => "1.1.1.1"
protocol => "http"
}
}

9

The configuration you've showed us is for receiving log messages. What is going to send events to the TCP ports you've defined here?

10

As it's mentioned, Windows event logs will be sent.

11

Yes, but what software component sends those messages? I don't believe Windows does it on its own.

12

I am using nxlog, here is the contents of conf file:

Please set the ROOT to the folder your nxlog was installed into,

otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

Module xm_json

Nxlog internal logs

Module im_internal Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();

Windows Event Log

# Uncomment im_msvistalog for Windows Vista/2008 and later Module im_msvistalog

Uncomment im_mseventlog for Windows XP/2000/2003

Module im_mseventlog

Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();

Module om_tcp Host 1.1.1. Port 5544

<Route 1>
Path internal, eventlog => out

13

Since you're using to_json() in your Nxlog config I'd expect you to want to use the json codec in your Logstash input configuration. But why not try it out? You'll see what codec changes (if any) need to be done.

(Hint: If you format config files as code they'll be sanely rendered and not like above.)

14

Yeah but that is only for NXLOG internal logs which I didn't configure in logstash yet because currently I am interested in Windows Events only; therefore, I have used module im_msvistalog in nxlog.conf.

By the way, which codec is recommended for Windows Events?

15

By the way, which codec is recommended for Windows Events?

This question doesn't make sense. Nxlog will read events regardless of source and turn them into its own internal key/value format, which can be rendered as plain text, JSON, or something else when e.g. sent over the network or written to a file. Nxlog's output codec should match Logstash's input codec, that's all.

17

Hi Again,

Thank you for your clarification. I got your point. Now, I am going to change the logging solution design little bit.

  • All the Windows Servers will be subscribed to Logstash server that is Windows based too.
  • I will install the nxlog on Logstash server that will rendered the local Windows events whereas local Windows events are being shipped from other servers via Windows default event subscription method.

The solution will be look like this (Windows Event Collector will serve as Logstash and nxlog):

What do you say about this solution? Any suggestion?

18

Sure, that looks reasonable. However:

  • It seems you need the non-free version of NXLog to collect eventlogs from remote Windows machines.

  • I'm not sure the XP-compatible im_mseventlog input supports pulling eventlogs from remote Windows machines.

19

Hi,

No, I will not pull logs from remote machines. U didn't get my point, Logstash is Windows based server which will act as events collector from other Windows machines. Then, I will install nxlog on Logstash server which will render the local logs.Remember, local logs are being pulled or pushed by other source Windows machines; therefore, nxlog doesn't need to pull logs from remote machines, Windows default event subscriber/publisher model will do instead.

20

could you please share link where I can download Logstash forwarder for Windows?
is there any Document , how to install in Windows?
is Logstash forwarder is works with WIndows server 2008R2?

thanks in advance

21

logstash-forwarder is deprecated in favor of Filebeat. If you have any follow-up questions about that please start a new thread.

1 Like