Remove white spaces from fields - Logstash

Elastic Stack Logstash
1

Message line:
key1=val1, key2=val2, key3=val3

Below is my logstash filter:

filter {

grok {
"match" => { "message" => "%{SYSLOGTIMESTAMP:msg_timestamp} %{GREEDYDATA:kvFields}"
}
}
kv {
source => "kvFields"
field_split => ","
trim_key => " "
trim_value => " "
}
}

Logstash Output:

{
"key1" => "val1",
" key2" => "val2",
" key3" => "val3"
}
Here i want to remove spaces in all fields. I tried with trim_key => "\s" in kv filter. still no luck.

Please help to remove spaces in all logstash fields if there is any other approach.

2

Have you tried

field_split => ", "

?

3

i am getting wrong output with this. In my message i have few fields which include " ". I need another approach. Found "strip " in mutate. But how to iterate it to apply this to all my fields?

4

i am getting wrong output with this. In my message i have few fields which include " ".

Please give an example.

5

my actualy msg line:

input message:
key1=first value key2=second value key3=value3

I did mutate gsub to make this ',' seperated kv msg line.

mutate {
gsub => ["kvFields", "(\S+=)", ", \1"]
}

now my new msg line is:
key1=first value, key2=second value, key3=value3

now i am applying kv filter on this msg line:
kv {
source => "kvFields"
field_split => ","
trim_key => " "
trim_value => " "
}

for this my output is:

{
"key1" => "first value",
" key2" => "second value"
" key3" => "third value"
}

But output fields containing spaces.

If i make field_split=", " i am getting output as:

{
"key1" => "first",
"key2" => "second"
"key3" => "third"
}

6

Ah, damn it. I forgot how field_split works. You can use a ruby filter to strip all keys and/or values without having to enumerate them in the configuration.

7

can u pls guide me how to loop all fields with ruby filter to apply strip on all keys? I am no way able to make this.

8

I tried this:

ruby {
code => '
event.each do |key, value|
mutate {
strip => [key]
}
end
'
}

Got syntax error:

SyntaxError: (ruby filter code):4: syntax error, unexpected tASSOC
strip => [key]
^

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.