Pattern for this log

[rojin]

Hello everyone! Can someone please help me with writing a grok pattern for this log line? I am not able to seperate its fields.

{"name":"test","test_arr":[{"key_word":"ok","test_code":"👌","h_code":"1234.png"}],"create_date":"2020-12-15 19:21:21.900","activation":true}

My problem is that I can not get the array fields. I am able to get "create_date" or "name" or "activation".
I'd be glad if someone could help me with this.

[Lukasz_Geras (Łukasz Geras)]

I would catch the fields outside of the brackets, and catch everything inside the square brackets as a single field.
Then, match everything inside brackets to the fields in separate grok filter, since grok can’create the nested fields.
But if you catch it this way, you can then use the rename filter, which works perfectly with nested fields.
Sorry, i didnt provide you the code, but im on the cellphone right now, i just finished creating my own question (it took 30 min on this bloody cellphone ) if you dont know what i mean, let me know i’ll provide you the example.

[rojin]

Hello Lukasz! Thank you for taking the time to answer me. I’d appreciate if you could provide the example so I can understand your solution better.

[Lukasz_Geras (Łukasz Geras)]

Two groks, and you have all parsed.

grok{
		match => {"message" => "^\{\"name\"\:"%{DATA:NAME}\"\,\"test_arr\"\:\[%{DATA:ARRAY}\]\,\"create_date\"\:\"%{DATA:TIME}\"\,\"activation\"\:%{DATA:ACTIVATED}\}$"}
		}
grok{
		match => {"ARRAY" => "key_word\"\:\"%{DATA:KEYWORD}\"\,\"test_code\"\:\"%{DATA:TEST_CODE}\"\,\"h_code"\:\"%{DATA:H_CODE}\""}
		}

Then, make the array

mutate{
		rename => ["KEYWORD" , "[ARRAY][KEYWORD]"]
		rename => ["TEST_CODE" , "[ARRAY][TEST_CODE]"]
		...
}

[rojin]

Thank you so much. it worked just fine (with a little bit of character replacement). I have a question: is there a way I can flatten the structured data? (I have removed fields such as "message", "ARRAY", "@timestamp"). Here's my output:

{
    "ACTIVATED" => "true",
     "@version" => "1",
        "ARRAY" => {
          "KEYWORD" => "ok",
        "TEST_CODE" => "👌"
    },
       "H_CODE" => "1234.png",
         "host" => "rojin.devops",
         "TIME" => "2020-12-15 19:21:21.900",
         "path" => "/home/rojin/Desktop/stickers/s.txt",
         "NAME" => "test"
}

Thanks again!

[Lukasz_Geras (Łukasz Geras)]

Just remove the mutate and you got it flat. Or did I misunderstood?

[rojin]

You are right! I thought the mutate part was for reaching nested fields. Thanks again!