Filter with Grok and KV


I'm trying to understand if I'm using the correct configuration for the following logs, or if there is a more efficient option for it.

2021-07-13T10:05:06.061Z <110>1 2021-07-13T08:46:58Z Keeper - 4269856080 [Keeper@Commander geo_location="Portland, Oregon, US" keeper_version_category="ADMIN" audit_event_type="login" keeper_version="Commander 15.4.85" username="" node_id="123456"] User logged in to vault

2021-07-13T10:05:06.061Z <110>1 2021-07-13T09:16:47Z Keeper - 4269938480 [Keeper@Commander geo_location="Vyshhorod, Kyivska oblast, UA" keeper_version_category="ADMIN" audit_event_type="enable_user" keeper_version="EMConsole 15.3.4" to_username="" username="" node_id="123456"] User test2@domain.comwas enabled by admin

filter {
   grok {
     match => { "message" => "%{GREEDYDATA:drop1}\[Keeper\@Commander%{GREEDYDATA:text}\]%{GREEDYDATA:description} "}
     remove_field => ["drop1", "message"]
   kv  {
     source => "text"
     trim_value => "\""

Thank you

If you do not want to keep a field there is no need to name it and the use remove_field. You could just use

match => { "message" => "%{GREEDYDATA}\[Keeper\@Commander%{GREEDYDATA:text}\]%{GREEDYDATA:description} "}

and if a pattern is not anchored it does not need to match the entire field. So you would be better off with

match => { "message" => "\[Keeper\@Commander%{GREEDYDATA:text}\]%{GREEDYDATA:description} "}

Personally I would do that using

match => { "message" => "\[Keeper\@Commander(?<text>[^\]]*)\]%{GREEDYDATA:description} "}

in case a one the kv pairs ever contains ]

It's very helpful, I'll use that, thank you!

I’m forwarding the output as a JSON, but it seems like for some reason, the last word is dropped from the description, even though, I can see it in the message itself. Any ideas what can cause it?

    "keeper_version":"KeeperEnterpriseBridge 15.1.0",
    "geo_location":"Tel Aviv, Tel Aviv, IL",
    "description":" User logged in to",
    "message":"<110>1 2021-07-15T07:03:07Z Keeper - 4278464624 [Keeper@Commander geo_location=\"Tel Aviv, Tel Aviv, IL\" keeper_version_category=\"ADMIN\" audit_event_type=\"login\" keeper_version=\"KeeperEnterpriseBridge 15.1.0\" username=\"\" node_id=\"123456\"] User logged in to vault"


The space at the end of your grok pattern indicate that the GREEDYDATA have to take each value until the last space of the field message. So it also don't take the last word after the space.


1 Like


Yep it did solve it,


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.