How to parse the json field with grok


(talonyuan) #1

HI here is my message

{"@timestamp":"2018-11-01T03:05:28.391Z","records":[{"properties":{"Keywords":0,"Message":"w3wp.exe Information: 0 :\r\n| 2018-10-31 02:43:16.396 | AuthorizeActivityAttributeBase:IsAuthorizedAsync | Line:63 | ActivityId:491b7b3e-61bc-490e-bcc0-286338e41727 | --> Entry","RoleName":"CloudPlatform.Tenant.Web","TraceEventId":0,"TraceMessage":"| 2018-10-31 02:43:16.396 | AuthorizeActivityAttributeBase:IsAuthorizedAsync | Line:63 | ActivityId:491b7b3e-61bc-490e-bcc0-286338e41727 | --> Entry","TraceSource":"w3wp.exe","RoleInstanceId":"CloudPlatform.Tenant.Web_IN_0","TraceEventType":8,"ProviderName":"SerilogInput","SourceContext":"SerilogTraceListener.SerilogTraceListener"},"level":"Informational","time":"2018-10-31T02:43:16.3968015+00:00"}],"@version":"1"}

now i want to grok records.properties.Message and get value of ActivityId

Here is my configuration in logstash

filter
{
grok
{
break_on_match => false
match => [
"[records][properties][Message]", "SystemGuid:(?.{37})",
"[records][properties][Message]", "ActivityId:(?.{37})"
]
}
}
}

but it doesn't work ,can you give me some advice
thank you


(Petr Simik) #2

I do similar staff with json + mutate filter

  1. #create parsed_json from input message
    
json {
    source => "message"
    target => "parsed_json"
     }
  1. #extract from parsed json important fields

mutate {

  add_field => {"timestamp" => "%{[parsed_json][timestamp]}"}
    add_field => {"message" => "%{[parsed_json][properties][Message]}"}
    add_field => {"ActivityId" => "%{[parsed_json][properties][ActivityId]}"}
  
    remove_field => [ "json", "message" ]
    remove_field => [ "json", "parsed_json" ]
  }

later you can parse ActivityId further if you like

match => { "ActivityId" => "(?(.{36}?))" }
or
match =>{ "ActivityId" => "(?(.*)) |" }


(talonyuan) #3

Hi
thanks for your response
here is my configuration and it work now.
filter
{
json{ source => "message" }
split{ field => "records" }
grok
{
break_on_match => false
match => [
"[records][properties][Message]", "SystemGuid:(?.{37})",
"[records][properties][Message]", "ActivityId:(?.{37})"
]
}
}


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.