Kv filter parsing

Elastic Stack Logstash
1

Is there a way to use KV filter for parse payload patterns as below:

A(1), B(2), C(3), D(T1)

Looked into config-option include_brackets, trim_key, trim_value

logstash filter is as below:

filter {
kv {
    value_split => "("
    trim_key => "\"\ \(\)"
    field_split => ","
    }
}

rubydebug as below:

elk_logstash     | {
elk_logstash     |       "@version" => "1",
elk_logstash     |        "message" => "A(1), B(2), C(3), D(T1), E(ttre)",
elk_logstash     |           "host" => {
elk_logstash     |         "name" => "3b015b1f625d"
elk_logstash     |     },
elk_logstash     |         "offset" => 0,
elk_logstash     |              "E" => "ttre)",
elk_logstash     |          "input" => {
elk_logstash     |         "type" => "log"
elk_logstash     |     },
elk_logstash     |              "D" => "T1)",
elk_logstash     |     "prospector" => {
elk_logstash     |         "type" => "log"
elk_logstash     |     },
elk_logstash     |              "B" => "2)",
elk_logstash     |     "@timestamp" => 2019-03-31T13:27:56.402Z,
elk_logstash     |              "A" => "1)",
elk_logstash     |              "C" => "3)",
elk_logstash     |         "source" => "/var/log/testfile",
elk_logstash     |           "beat" => {
elk_logstash     |          "version" => "6.5.2",
elk_logstash     |         "hostname" => "3b015b1f625d",
elk_logstash     |             "name" => "3b015b1f625d"
elk_logstash     |     },
elk_logstash     |           "tags" => []
elk_logstash     | }
2

You need to add trim_value

    trim_value => "\)"
3

@Badger thanks. I was able to figure out.

Some of the value contain , , and with field_split it causes a problem. Is there a way to condition it.
Example:
A(1), B(2), C(3), D(1,2), E(), F(12)

4

You could try

mutate { gsub => [ "message", "(\([^()]*),(([^()]*))", "\1|\2" ] }

which, if it finds a , inside () changes it to |, so you end up with

   "message" => "A(1), B(2), C(3), D(1|2), E(), F(12)"

which can be parsed using

    kv {
        field_split => ","
        value_split => "("
        trim_key => "\"\ \(\)"
        trim_value => "\)"
    }
5

Yep, worked. Added gusb again to revert the | to , after the kv filter. Thanks.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.