How to get kv filter to ignore value_split in data

[Nofar]

My conf file:

kv {
  source => "cp_keyValueData"
  field_split => ";"
  value_split => ":"
  trim_key => " "
}

data example:

key1: val1; key2: val2; key3:  https://site/?g={......"...;%20%20CLR%20;%20rv:11.0)"..}; key4: val4;

it split to:

key1:             val1
key2:             val2;
key3:            https://site/?g={......"...;%20%20CLR%20;
%20rv:         11.0)"..};
key4:            val4;

How can i prevent this mistake?

There is some way to ignore value_split that found in string?

thank u !

[Mojster]

I've have had a similar problem.
My solution was to implement my own splitter in Ruby.

Here's my topic:
How to handle '=' in values, splitting on | but KV takes over all '=' not only the first

But as I see, you have a problem with pair split parameter';', mine was with key/value split parameter '='.
Hope you're can use mine as a waypoint.

[guyboertje]

Your issue is not with the value_split, it is with the field_split of ;.

You have two options:

1. If your values are always percent encoded when it contains a ; e.g. https://site/?g={......"...;%20%20CLR%20;%20rv:11.0)"..}, then inside the value you will never see ; i.e. "semi-colon space" so you can make that your field_split value.
2. If your values are not percent encoded and contains a ; then your only option is to use mutate gsub but not simply to replace the ; because that will replace the semi-colon in the values too. You will need know all the possible keys and gsub for them but also to use a named pattern to capture the found key and substitute it back.

Example:

input {
  generator {
    message => 'key1: val1; key2: val2; key3:  https://site/?g={......"...;  CLR  rv:11.0)"..}; key4: val4;'
    count => 1
  }
}

filter {
  mutate {
    gsub => ["[message]", ";\s*(?<key>key1|key2|key3|key4)", '|^|\k<key>']
  }
  kv {
     field_split => "|^|"
     value_split => ":"
     source => "message"
  }
}

output {
  stdout {
    codec => rubydebug {metadata => true}
  }
}

Result:

{
          "key1" => "val1",
      "sequence" => 0,
          "key2" => "val2",
    "@timestamp" => 2018-01-17T11:57:10.574Z,
          "key3" => "https://site/?g={......\"...;  CLR  rv:11.0)\"..}",
          "key4" => "val4;",
      "@version" => "1",
          "host" => "Elastics-MacBook-Pro.local",
       "message" => "key1: val1|^|key2: val2|^|key3:  https://site/?g={......\"...;  CLR  rv:11.0)\"..}|^|key4: val4;"
}