How to Split the one field into multiple fields using kv plugins

Elastic Stack Logstash
4

If an example of log_message is

(Institution Id->77) (data Id->127) QUERY-> insert into tablename1(col1,col2) values(val1,val2) ERROR->Table 'tablename1' doesn't exist

Then you could parse that (and the other 2) using this

    grok { match => { "log_message" => "^%{DATA:prefix}QUERY\-> %{DATA:query} ERROR(: |\->)%{DATA:error}$" } }
    if [prefix] {
        ruby {
            code => '
                m = event.get("prefix").scan(/\(([^-]+)\->([^\)]+)\)/)
                m.each.each { |k, v|
                    event.set(k, v)
                }
            '
        }
    }

In the ruby filter the scan function matches a regexp against the "prefix" field. The regexp looks for (key->value) patterns and returns an array of them. Each match is on entry in the array, and that entry itself is an array, the first entry of which is the key, and the second is the value.

2 Likes