How to put two groks for IIS Logs?

Elastic Stack Logstash
1

<grok
{
break_on_match => true
match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:request}(?:%{URIPARAM:requestparam})? - %{NUMBER:port} - %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"]
match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:cs_uri_stem} - %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"]

	}>
2

If you want to match against two patterns then use

grok {
    match => {
        "message" => [
            "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:request}(?:%{URIPARAM:requestparam})? - %{NUMBER:port} - %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}",
            "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:cs_uri_stem} - %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"
        ]
    }
}
3

This is my Logstash file, am I doing it wrong?

<input
{
file
{
path => [
"C:/Elastic_stack/logs/IISLogs/u_ex.log"
]
start_position => "beginning"
sincedb_path => "NUL"

}

}
filter
{
if "logs" in [path]
{

	grok {
		match => {
			"message" => [
							"%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:request}(?:%{URIPARAM:requestparam})? - %{NUMBER:port} - %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}",
							"%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:method} %{URIPATH:cs_uri_stem} - %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clienthost} %{NOTSPACE:useragent} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"
						]
		}
	}
	

	# set the event timestamp from the log
	date 
	{
		match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]
		timezone => "Etc/UCT"
	}

	# matches the big, long nasty useragent string to the actual browser name, version, etc
	mutate 
	{
		remove_field => [ "log_timestamp"]
	}
}
fingerprint 
{
	id 		=> "ApplicationLogs"
	source 	=> ["@timestamp","message"]
	target 	=> "[fingerprint]"
	key 	=> "78787878"
	method 	=> "SHA1"
	concatenate_sources => true
}

}
output
{
stdout
{
codec => rubydebug
}
if "IISLogs" in [path]
{
elasticsearch
{
hosts => ["http://localhost:9200"]
document_id => "%{[fingerprint]}"
index => "u_ex_pl_sim-%{+YYYY.MM.dd}"
}
}
}>

4

That looks reasonable, what do you not like about the output?

BTW, when posting configurations please select the configuration and use the </> button in the toolbar above the edit pane. That will preserve the formatting of the configuration. Notice how in your last post the formatting of the grok and date filters are preserved, but the formatting of the input section is not. If you use </> the formatting is preserved everywhere, which makes the configuration much easier to read.

5

<

image
image1512×552 42.7 KB
>

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.