# Filter with Grok and KV

**URL:** <https://discuss.elastic.co/t/filter-with-grok-and-kv/278697>\
**Category:** Logstash\
**Created:** [July 14, 2021, 5:39pm UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697 "2021-07-14T17:39:15Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sunflower](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@Sunflower](https://discuss.elastic.co/u/Sunflower)\
**Post date:** [July 14, 2021, 5:39pm UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697/1 "2021-07-14T17:39:15Z")

</div>

Hi,

I'm trying to understand if I'm using the correct configuration for the following logs, or if there is a more efficient option for it.

> 2021-07-13T10:05:06.061Z 10.20.30.40 \<110\>1 2021-07-13T08:46:58Z 44.236.133.39 Keeper - 4269856080 [Keeper@Commander geo\_location="Portland, Oregon, US" keeper\_version\_category="ADMIN" audit\_event\_type="login" keeper\_version="Commander 15.4.85" [username="test@domain.com](mailto:username=%22test@domain.com)" node\_id="123456"] User [test@domain.com](mailto:test@domain.com) logged in to vault

> 2021-07-13T10:05:06.061Z 10.20.30.40 \<110\>1 2021-07-13T09:16:47Z 77.52.201.194 Keeper - 4269938480 [Keeper@Commander geo\_location="Vyshhorod, Kyivska oblast, UA" keeper\_version\_category="ADMIN" audit\_event\_type="enable\_user" keeper\_version="EMConsole 15.3.4" [to\_username="test2@domain.com](mailto:to_username=%22test2@domain.com)" [username="test3@domain.com](mailto:username=%22test3@domain.com)" node\_id="123456"] User test2@domain.comwas enabled by admin [test3@domain.com](mailto:test3@domain.com)

```auto
filter {
   grok {
     match => { "message" => "%{GREEDYDATA:drop1}\[Keeper\@Commander%{GREEDYDATA:text}\]%{GREEDYDATA:description} "}
     remove_field => ["drop1", "message"]
  }
   kv {
     source => "text"
     trim_value => "\""
     }
}

```

Thank you

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 14, 2021, 5:52pm UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697/2 "2021-07-14T17:52:08Z")

</div>

> [@Sunflower](#):
>
> ```auto
> match => { "message" => "%{GREEDYDATA:drop1}\[Keeper\@Commander%{GREEDYDATA:text}\]%{GREEDYDATA:description} "}
> 
> ```

If you do not want to keep a field there is no need to name it and the use remove\_field. You could just use

```auto
match => { "message" => "%{GREEDYDATA}\[Keeper\@Commander%{GREEDYDATA:text}\]%{GREEDYDATA:description} "}

```

and if a pattern is not anchored it does not need to match the entire field. So you would be better off with

```auto
match => { "message" => "\[Keeper\@Commander%{GREEDYDATA:text}\]%{GREEDYDATA:description} "}

```

Personally I would do that using

```auto
match => { "message" => "\[Keeper\@Commander(?<text>[^\]]*)\]%{GREEDYDATA:description} "}

```

in case a one the kv pairs ever contains ]

---

<div class="post-metadata">

**Author:** ![Sunflower](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@Sunflower](https://discuss.elastic.co/u/Sunflower)\
**Post date:** [July 14, 2021, 6:01pm UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697/3 "2021-07-14T18:01:06Z")

</div>

It's very helpful, I'll use that, thank you!

---

<div class="post-metadata">

**Author:** ![Sunflower](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@Sunflower](https://discuss.elastic.co/u/Sunflower)\
**Post date:** [July 15, 2021, 2:06pm UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697/4 "2021-07-15T14:06:42Z")

</div>

I’m forwarding the output as a JSON, but it seems like for some reason, the last word is dropped from the description, even though, I can see it in the message itself. Any ideas what can cause it?

```auto
 {
    "type":"keeper",
    "@version":"1",
    "keeper_version":"KeeperEnterpriseBridge 15.1.0",
    "username":"user@domain.com",
    "audit_event_type":"login",
    "geo_location":"Tel Aviv, Tel Aviv, IL",
    "port":40236,
    "@timestamp":"2021-07-15T07:08:05.795Z",
    "node_id":"123456",
    "keeper_version_category":"ADMIN",
    "host":"10.20.30.40",
    "description":" User user@domain.com logged in to",
    "message":"<110>1 2021-07-15T07:03:07Z 1.2.3.4 Keeper - 4278464624 [Keeper@Commander geo_location=\"Tel Aviv, Tel Aviv, IL\" keeper_version_category=\"ADMIN\" audit_event_type=\"login\" keeper_version=\"KeeperEnterpriseBridge 15.1.0\" username=\"user@domain.com\" node_id=\"123456\"] User user@domain.com logged in to vault"
 }

```

---

<div class="post-metadata">

**Author:** ![Cad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cad/32/86661_2.png) [@Cad](https://discuss.elastic.co/u/Cad)\
**Post date:** [July 15, 2021, 3:18pm UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697/5 "2021-07-15T15:18:22Z")

</div>

Hi,

The space at the end of your grok pattern indicate that the GREEDYDATA have to take each value until the last space of the field message. So it also don't take the last word after the space.

Cad

---

<div class="post-metadata">

**Author:** ![Sunflower](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@Sunflower](https://discuss.elastic.co/u/Sunflower)\
**Post date:** [July 18, 2021, 6:52am UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697/6 "2021-07-18T06:52:22Z")

</div>

Hi,

Yep it did solve it,

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 15, 2021, 6:52am UTC](https://discuss.elastic.co/t/filter-with-grok-and-kv/278697/7 "2021-08-15T06:52:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.