KV Plugin Not working

[premkumar]

Hi All,

I have been trying to extract key value pairs from a certain log message.

Log Message Sample
a=a1, b=b1, c=c1, d=d1,d2, e=e1 hello world, f=f1

Expected Output
a=a1
b=b1
c=c1
d=d1,d2
e=e1
f=f1

Actual Output
a=a1
b=b1
c=c1
d=d1
e=e1 hello world
f=f1

KV Plugin
kv {
source => "body"
field_split => ","
value_split => "="
trim_value => "},"
tag_on_failure => ["kv-parse-failed"]
}

Need help on this. Have been trying for a very long time.
Not sure whether to use any other plugin to obtain the expected result.

[Badger]

You can use

field_split_pattern => ", "

to define a two character field delimiter. That will result in d being parsed the way you want.

[premkumar]

@Badger, it did not work out

[Christian_Dahlqvist]

You expected output is not the expected one when using the kv filter. The whole string is parsed which means hello world will need to go somewhere.

[premkumar]

But, is it possible to remove the hello world after the e field is extracted?

[sjabiulla (JB)]

Yes, you can use mutate filter to replace hello world with empty string

So, you will have it something like below. Below code should remove hello world from e field. Place the below filter after your kv filter.

mutate {
    gsub => [
      "e", " hello world", "",
    ]
  }

[premkumar]

Hi @sjabiulla,
The mutate filter is not working out. Tried it multiple times.

[sjabiulla (JB)]

Can you please show the config that you have tried with mutate filter and it's better to share your whole config, so that we can try this side and provide the solution

[premkumar]

I have found out the issue. Need to use the mutate filter plugin before the kv filter plugin.