Kv filter's behavior when square brackets in log data

(Pranav Dixit) #1

In logstash I am using the following filter-

filter {
kv {
field_split => "\t"

So here I am using tab as field split and the value split I have kept default (i.e. '=')

Now I am sending a log like this-

animal=cat fruit=[apple]banana

where there is a tab between cat and fruit.

This is producing the filtered output as-

"animal": "cat"
"fruit": "apple"
"message": "animal=cat\tfruit=[apple]banana"

But I was expecting - "fruit": "[apple]banana".

Also then if I send a log like this-

animal=cat fruit=[apple]banana=healthy

where tab is only between cat and fruit
then the output produced is -

"banana" : "healthy"
"animal": "cat"
"fruit": "apple"
"message": "animal=cat\tfriut=[apple]banana=healthy"

But here I was expecting - "fruit": "[apple]banana=healthy" and no separate key as 'banana'.

Is this a bug or am I missing something here?

(João Duarte) #2

The kv filter works with many many regular expressions, and characteres like [|]<> can severely interfere with how the plugin works. There are even options to remove these characters if you know in advance they appear in the data

I suggest maybe using mutate gsub operations to make the text string more uniform so that the kv filter doesn't have to guess (and make the wrong decisions).

(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.