Parsing message using kv filter

Elastic Stack Logstash
1

Hi all,

I'd like to parse some part of a message with VKv filter, and keep end of the message into another field.

Message is :
[MarketplaceGroup=data][MarketplaceId=10][AccountId=1234] updating 926 items

My filter is :
filter {
kv {
source => "message"
field_split_pattern => "\]\["
trim_key => "\[\]"
trim_value => "\[\]"
}
}

Result I have is :
{
"MarketplaceGroup" => "data",
"@timestamp" => 2020-02-13T17:15:30.615Z,
"MarketplaceId" => "10",
"AccountId" => "1234] updating 926 items",
"message" => "[MarketplaceGroup=data] [MarketplaceId=10] [AccountId=1324] updating 926 items"
}

result I'd like to have is :

{
"MarketplaceGroup" => "data",
"@timestamp" => 2020-02-13T17:15:30.615Z,
"MarketplaceId" => "10",
"AccountId" => "1234",
"message" => "updating 926 items"
}

I can't find how to define all my key/values are inside braces, and rest of field is what I want to keep in message field (or store in another field, whatever)

thanks
thomas

2

I would

mutate { copy => { "message" => "otherMessage" } }
mutate { gsub => [ "message", "[^\]]*$", "", "otherMessage", ".*\]([^\]]*)$", "\1" ] }

then kv the [message] field.

3

Ah, regex magic, it works like a charm!
thanks badger

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.