Grok filter by values

Elastic Stack Logstash
1

Hi Everyone,

I have a text in my input as follows,

The following text contains {"Food":"Fruit","Type":"Apple"}

I am trying to structure my output data to also contain fields

Food: Fruit
Type: Apple

I was looking into grok filter, to help me achieve this.

I understand there are already some patterns in place to extract other log details such as timestamp, ip etc.

I also have looked into custom patterns (which uses regex)

I am just wondering if it is possible to use custom patterns to extract these fields e.g. Fruit and Type.

Or I should be looking into some other filter operation instead of using grok.

As you can see; they are all string values

Thanks

2

something I can quickly think on this is to parse message like

grok {
 %{DATA:log.initial_text}\{%{DATA:key_values_string}\}
}

so you will have key_values_string = "Food":"Fruit","Type":"Apple"
and you will be able to parse it with kv filter

3

I belive best solution could be

  1. use grok to get json in separate field
    grok {
        match => {
            "message" => ["%{DATA:log.initial_message}%{JSONOBJ:json_body}"]
        }
        pattern_definitions => {
            #JSONOBJ {.*$
            "JSONOBJ" => "{.*$"
        }
    }
  1. use json filter to parse it
json {
    source => "json_body"
}

image
image973×776 17.5 KB

Cheers!

4

Yeah that seems to be the solution to extract the json body out.
I am playing with other patterns to suit by input message.
Thanks :smiley:

5

You probably don't want to use the grok filter to parse the contents of the JSON. Simply extract the JSON into a new value as in the above example. Then use the json filter to unmarshal the JSON value into fields.