Grok filter for different kinds of log in a single file

Elastic Stack Logstash
1

Hi,

I have written grok filter for 2 different types of log lines in a single file.
The output is not coming but.

The filter that I have written:


filter {
    grok {
        match => {
            "message" => "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Querystring: %{WORD:req_type}&pVersion=%{WORD:pversion}&contRep=%{WORD:rep_type}" 
			
        }
    }
	grok {
        match => {
            
			"message" => "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Path: /isap/archivelink/%{GREEDYDATA:holding_type}"
        }
    }
	
	date {
				match => [ "logdate", "YYYY-MM-dd HH:mm:ss,SSS" ]
				target => "logdate"
		}
		
		if "_grokparsefailure" in [tags]{
			drop{ }
		}
		
		
}

Here is the part of log file.
The two lines for which filter is written.

[2018-06-17 20:41:17,864|INFO |https-kite-100|com.defghu.jjj|service|293 ]	Querystring: fun&pVersion=0046&contRep=W9&docId=00215A9
[2018-06-17 20:41:50,329|INFO |https-joy-88|com.abcd.efg|service|292 ]	Path: /iasap/archivelink/fun_game

I am not getting output.
Can anyone please help me what I am doing wrong?

2

try using grok debugger in kibana dev tools or grokdebug.herokuapp.com to see whether your grok pattern matches your log

3

You have two groks that cannot both match the same line, so if one matches the other will always result in a _grokparsefailure, so you will drop {} every line. You could try

grok {
    match => {
        "message" => [
            "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Querystring: %{WORD:req_type}&pVersion=%{WORD:pversion}&contRep=%{WORD:rep_type}",
            "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Path: /isap/archivelink/%{GREEDYDATA:holding_type}"
        ]
    }
}
4

Thank you,It worked.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.