Windows event logs, NXLog, and Logstash - Oh My!


(Edrick Goad) #1

I am trying to forward the event logs from my Domain Controllers into Logstash, but something isnt working. I have followed many of the examples out there, but can't get it going.

Environment: Ubuntu 15.04 server, Elasticsearch 2.1, Logstash 2.1, and Kibana 4.3. I am forwarding from a Windows 2012 server, with nxlog-ce-2.8.1248.

I am at a loss of what to try next and any help would be appreciated. If NXLog isnt the right tool for the job, please direct me to something better.

NXLog config file

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension syslog>
 Module xm_syslog
</Extension>
<Extension json>
Module      xm_json
</Extension>

<Input in>
Module      im_msvistalog
	Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
# For windows 2003 and earlier use the following:
#   Module      im_mseventlog
</Input>



<Output out>
Module      om_tcp
Host        MCCELK.meteorcomm.lan
Port        3515
</Output>


<Route 1>
Path        in => out
</Route>

Logstash config file

input {
# Accept messages in on tcp/3515
# Incoming messages will be in json format, one per line
# Tag these messages as windows and eventlog so we can filter on them later on
tcp {
port => 3515
#        codec => json_lines { charset => CP1252 }
codec => "line"
tags => ["windows","eventlog"]
type => "WindowsEventLog"
}
}
filter{
if [type] == "WindowsEventLog" {
json{
source => "message"
}
if [SourceModuleName] == "eventlog" {
mutate {
replace => [ "message", "%{Message}" ]
}
mutate {
remove_field => [ "Message" ]
}
}
}
}


output {
  elasticsearch { hosts => ["localhost:9200"] }
#  stdout { codec => rubydebug }
}

(Magnus Bäck) #2

Comment out the elasticsearch output and uncomment the stdout output. Do things one step at a time.

Is Logstash getting any connections at all? Is there anything in the NXLog logfile? Is there anything in the Logstash logfile? What if you crank up logging with --verbose or even --debug?


(Edrick Goad) #3

Example event can be found at http://dumptext.com/m4Zxq18K


(Magnus Bäck) #4

Okay, but the first of those error messages came from Elasticsearch. Please disable that output for now.


(system) #5