Working with key:value arrays

Elastic Stack Logstash
4

Your values contain commas, so a simple mutate+split will incorrectly parse the [message]. Luckily, a kv filter parses it correctly

kv { field_split => "," value_split => ":" trim_key => '"' target => "[@metadata][kvData]" }
    ruby {
        code => '
            kvData = event.get("[@metadata][kvData]")
            if kvData
                data = []
                kvData.each { |k, v|
                    data << { "Field1" => k, "Field2" => v }
                }
                event.set("someField", data)
            end
        '
    }
    split { field => "someField" }

You can move the results to the top level using this code.

I think using a kv filter is fragile, and I would do it in ruby.

    ruby {
        code => '
            matches = event.get("message").scan(/"([^"]*)":"([^"]*)"(,|$)/)
            data = []
            matches.each_index { |x|
                data << { "Field1" => matches[x][0], "Field2" => matches[x][1] }
            }
            event.set("someField", data)
        '
    }
    split { field => "someField" }

Although that regexp may not be any less fragile than the kv filter :smiley:

1 Like