Grok filter when logs format is not fixed

Elastic Stack Logstash
1

Am trying to parse the rest api request and response in my node logs. Below are two example logs
{"level":"debug","message":"HTTP GET /1stexample/alerts/subs?userId=111","meta":{"res":{"statusCode":200},"req":{"headers":{"user-agent":"PostmanRuntime/1.1.1","accept":"/","cache-control":"no-cache","postman-token":"11111-5291-2222-b450-0d15579922206","accept-encoding":"gzip, deflate","via":"1.1 IDCFPCLUSTER 0A83490F ","host":"10.20.30.40:8888"}},"responseTime":59}}

{"level":"debug","message":"HTTP GET /2ndexample/alerts/subs?userId=111","meta":{"res":{"statusCode":200},"req":{"headers":"accept":"/","cache-control":"no-cache","postman-token":"11111-5291-2222-b450-0d15579922206","accept-encoding":"gzip, deflate","via":"1.1 IDCFPCLUSTER 0A83490F ","host":"10.20.30.40:8888"}},"responseTime":59}}

Am able to write a grok filter parse it and make sense out of it in kibana. But when there is a slight change in log , as shown in second log user-agent is missing, by grok parsing fails.

How to handle such cases?

2

That looks like JSON, so use a JSON filter instead of grok.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.