Need Help to extract custom log data using gork

We want to extract some custom logs like this,

[2022-10-19 07:45:21] test_site.INFO: REQUEST: [{"username":"6HANT","password":"u5469!230","grant_type":"password"},{"Content-Type":"application/json"},"https://abc.SD.com:8080/retin/api/token","POST"] []

[2022-10-19 07:45:21] test_site.INFO: RESPONSE:{"access_token":"eyJhbGciOiJIIkpXVCJ9.eyJ1bmlxdWE2NjYx2NjE2ODUy1NTIxfQ.1rvspnSrqeW3w9vNoZmWqEcxW156w","token_type":"bearer","expires_in":"5"} [] []

[2022-10-19 07:45:21] test_site.INFO: REQUEST: [{"RequestData":{"MobeNum":"+880657567324"},"RequestId":"1660","ChannelId":"ShYZ"},{"Content-Type":"application/json","Authorization":"Bearer eyJhbVCJ9.eyJ1bmlx1MjEsMSwiaWF0IjoTIxfQ.1rvx2x0ZLbW156w"},"https://dev.abc.com:8080/refin/api/acctpen/genep","POST"] []

We are interested only json data as separated fields and url.

any one have any clue please help me. Thank's in advance.

Hi,

First you can use the grok debugger in Kibana dev tools

https://KIBANA:5601/app/dev_tools#/grokdebugger

You can put your data into the "sample data" form, when we look at your data we can see that there is weird JSON arrays

Here is a very basic one that will help you to get further

(?(.+?]))%{SPACE}(?(\w+\S+))%{SPACE}(?(\w+\S+))%{SPACE}(?<json_message>(.+))

( Really dirty regex i dont really took the time )

To help you more https://regex101.com/ And try tit yourself !