Kv filter usage

[PaoloZhao]

Hi guys!
I am a fresh fish.
I have some logs like blow.

15:08:16.2104 Info {"message":"BlankProcess execution started","level":"Information","logType":"Default","timeStamp":"2021-09-21T15:08:16.2015207+08:00","fingerprint":"1ac6b349","windowsIdentity":"abc","machineName":"123","processName":"BlankProcess","processVersion":"1.0.0","jobId":"4e65df55-6001-4f4c-aa90-910346e5e0eb","robotName":"abc","machineId":0,"fileName":"Main","initiatedBy":"Studio"}
15:08:18.2199 Error {"message":"Throw: Test","level":"Error","logType":"Default","timeStamp":"2021-09-21T15:08:18.2199587+08:00","fingerprint":"bb6488d3","windowsIdentity":"abc","machineName":"123","processName":"BlankProcess","processVersion":"1.0.0","jobId":"4e65df55-6001-4f4c-aa90-910346e5e0eb","robotName":"abc","machineId":0,"fileName":"Main"}
15:08:18.2748 Info {"message":"BlankProcess execution ended","level":"Information","logType":"Default","timeStamp":"2021-09-21T15:08:18.2738121+08:00","fingerprint":"61ab270f","windowsIdentity":"abc","machineName":"123","processName":"BlankProcess","processVersion":"1.0.0","jobId":"4e65df55-6001-4f4c-aa90-910346e5e0eb","robotName":"abc","machineId":0,"totalExecutionTimeInSeconds":2,"totalExecutionTime":"00:00:02","fileName":"Main"}

I thinks that It should be used kv filter.
before kv filter need extract string between {}.
How can I do this, who can give me a sample for that.

Best Regards!

[leandrojmp (Leandro Pereira)]

There is no kv in your message, you have a static part and a json part.

You can parse it using dissect to split your message in three different fields, where the last one will be a json, then you use a json filter to parse this last field.

You need something like this.

filter {
  dissect {
    mapping => {
      "message" => "%{timestamp} %{loglevel} %{jsonMsg}"
    }
  }
  json {
    source => "jsonMsg"
  }
}

The dissect filter will create three fields, using your first message as an example you will have.

timestamp: 15:08:16.2104
loglevel: Info
jsonMsg: {"message":"BlankProcess execution started","level":"Information","logType":"Default","timeStamp":"2021-09-21T15:08:16.2015207+08:00","fingerprint":"1ac6b349","windowsIdentity":"abc","machineName":"123","processName":"BlankProcess","processVersion":"1.0.0","jobId":"4e65df55-6001-4f4c-aa90-910346e5e0eb","robotName":"abc","machineId":0,"fileName":"Main","initiatedBy":"Studio"}

The json filter will parse your jsonMsg field and extract the fields in the root of the document.

[PaoloZhao]

Hi @leandrojmp
Thanks for your reply!
It's perfect!