Grok filter for different kinds of log in a single file

Hi,

I have written grok filter for 2 different types of log lines in a single file.
The output is not coming but.

The filter that I have written:


filter {
    grok {
        match => {
            "message" => "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Querystring: %{WORD:req_type}&pVersion=%{WORD:pversion}&contRep=%{WORD:rep_type}" 
			
        }
    }
	grok {
        match => {
            
			"message" => "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Path: /isap/archivelink/%{GREEDYDATA:holding_type}"
        }
    }
	
	date {
				match => [ "logdate", "YYYY-MM-dd HH:mm:ss,SSS" ]
				target => "logdate"
		}
		
		if "_grokparsefailure" in [tags]{
			drop{ }
		}
		
		
}

Here is the part of log file.
The two lines for which filter is written.

[2018-06-17 20:41:17,864|INFO |https-kite-100|com.defghu.jjj|service|293 ]	Querystring: fun&pVersion=0046&contRep=W9&docId=00215A9
[2018-06-17 20:41:50,329|INFO |https-joy-88|com.abcd.efg|service|292 ]	Path: /iasap/archivelink/fun_game

I am not getting output.
Can anyone please help me what I am doing wrong?

try using grok debugger in kibana dev tools or grokdebug.herokuapp.com to see whether your grok pattern matches your log

You have two groks that cannot both match the same line, so if one matches the other will always result in a _grokparsefailure, so you will drop {} every line. You could try

grok {
    match => {
        "message" => [
            "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Querystring: %{WORD:req_type}&pVersion=%{WORD:pversion}&contRep=%{WORD:rep_type}",
            "%{TIMESTAMP_ISO8601:logdate}\|%{WORD:service_type} \|%{DATA:url}\|%{DATA:filename}\|%{WORD:request_type}\|%{INT:l_num} \]	Path: /isap/archivelink/%{GREEDYDATA:holding_type}"
        ]
    }
}

Thank you,It worked.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.