Grok filter for different type of log lines

Hi,

I am shipping data from logstash to Elasticsearch.

Can you please help me to write the grok filter .
I have attached a part of the log files.

[2018-06-17 20:12:59,754|INFO |https-jkilhhs|com.myfirst.service.ai.fw|service|292 ]	Path: /iasap/archilink
[2018-06-17 20:12:59,754|INFO |https-jsse|com.myfirst.service.ai.fw|service|293 ]	Querystring: get&pVersion=0046&contRep=W9&docId=009B0669&compId=data&fromOffset=12279&toOffset=124&accessMode=r&authId=CN%3DU
[2018-06-17 20:12:59,754|INFO |https-jsse|com.myfirst.service.ai.fw|acceptRequest|28  ]	Starting request of type get
[2018-06-17 20:12:59,754|INFO |https-jsse|com.myfirst.service.ai.fw|acceptRequest|43  ]	type=get, state=CheckSyntax

You can use the following grok filter to parse. I couldn't capture the entire line as I do not the information you would like to capture from these log lines. But, you can use the below grok filter as a starting point.

match => {"message" => "%{DATESTAMP:DATE}[|]INFO [|]https-%{WORD:variable1}[|]com.myfirst.service.ai.fw[|]%{WORD:variable2}[|]%{INT:variable3} .\t%{GREEDYDATA}"

You can try something similar to the following custom pattern and grok pattern if you want to capture all fields.

Custom Patterns

WORDHYPHEN [a-zA-Z\-]+
LOG_MODULE [a-zA-Z\-0-9\.]+

Grok Pattern

\[%{TIMESTAMP_ISO8601:timestamp}\|%{WORD:log_level} \|%{WORDHYPHEN:process}\|%{LOG_MODULE:module}\|%{WORD:log_type}\|%{NUMBER:num1}\s+\]\s+%{GREEDYDATA:message}

I want to capture Querystring : get and contRep=W9.
I have attached a part of log file.
I the log file there are different values for Querystring and contRep.
For ex: for Querystring values are get,create,update.
contRep values are W9,W6 and so on.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.