Not getting any ouput

Elastic Stack Logstash
1

I'm trying to see the output of an if statement with grok filters. Logstash says it's starting however there is no file ever created in my output. Shouldn't I be seeing something even if there arent any matches?

Test input:
{"message":"<36>Nov 02 15:48:57 LCE: [matched] 1.1.1.1:0 -> 1.1.1.1:0 :: Microsoft-Windows-WMI-Activity/Operational,11/02/2015,15:48:00 PM,Microsoft-Windows-WMI-Activity,5858,Error,N/A,None,N/A,GVLRESCMA02.shermfin.com,IP:10.36.48.32,5858,Id = {074A806D-1260-000E-6F80-4A076012D101}; ClientMachine = SERVERNAME; User = NT AUTHORITY\SYSTEM; ClientProcessId = 1964; Component = Unknown; Operation = Start IWbemServices::ExecQuery - ROOT\STANDARDCIMV2 : ASSOCIATORS OF {MSFT_NetLbfoTeamMember.InstanceID="{DE3CFAF2-9030-431E-8CDC-007673D0C50E}"} WHERE ResultClass=MSFT_NetLbfoTeam; ResultCode = 0x80041008; PossibleCause = Unknown ","@version":"1","@timestamp":"2015-11-02T20:48:57.843Z","type":"LCE","host":"1.1.1.1"}

input {
  file{
	path => "C:\ELK\running\logstash-2.0.0\test\testinputgrok36.txt"
	#type => "LCE"
	}
}
filter {
	grok{
  		match => {"message" => "<%{BASE10NUM:LCE_log_num}>%{SYSLOGTIMESTAMP:LCE_time} %{NOTSPACE}: %{NOTSPACE} %{IP:Source_IP}:%{BASE10NUM:Source_Port} -> %{IP:Destination_IP}:%{BASE10NUM:Destination_Port} ::%{GREEDYDATA:Message_Data}" }
  		add_field => { "sort_num" => "%{LCE_log_num}" }
	}
if [sort_num] == "36" {
	grok{
		match => [ "Message_Data", "%{PROG:Log_Type},%{DATE_US:Event_Date},%{TIME:Event_Time} %{WORD},%{PROG:Log},%{BASE10NUM},%{WORD:Error_id},%{PROG}%{SPACE}%{PROG},%{WORD},%{PROG},%{HOSTNAME},IP:%{IP},%{BASE10NUM}%{GREEDYDATA:Win_Log}" ]
		add_tag => [ "suceeded grok" ]
	}
}
else {
	grok{
		add_tag => "failed grok"
	}
}
}
output {
  file{
    path => "C:\ELK\running\logstash-2.0.0\test\groktest.txt"
  }
}

Any and all help would be extremely appreciated.

Thanks

2

Logstash is waiting for more data to be appended to testinputgrok36.txt, which it's tailing. For testing purposes like this I suggest you use the stdin input and redirect your test file to Logstash.

3

Thanks, What I actually ended up doing was added start_postion => ["beginning"] to the input and that has allowed me to test. Thanks for the input though!!

4

That'll still only allow you to test it once. The second time you run Logstash with the exact same file Logstash will continue to tail it. This is a major source of confusion among Logstash beginners, so beware.

5

How do i use STDIN with a file?

6 
command < file

which (on Windows) is equivalent to

type file | command
7

I don't understand this. Can you show me the syntax relative to my above example?

8 
C:\path\to\logstash\executable -f C:\path\to\configfile < C:\ELK\running\logstash-2.0.0\test\testinputgrok36.txt
9

Thanks again!!! Is there a way when writing a GROK filter you can just say i dont care what data is in between x and y?

for example:
A new process has been created. <---I want this
Subject: Security ID: S-1-5-18 <---I don't want this
Account Name: XXXXX <---I want this

10

Are those three lines part of the same multi-line message or are they three different messages?

11

They are all in the same line

12

The grok filter doesn't do search-and-replace (use gsub for that), but grok is usually used for extracting fields from a larger message and you can choose to simply not extract the contents of the Subject line.