# Grok: multiple occurences of pattern

**URL:** <https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957>\
**Category:** Logstash\
**Created:** [March 28, 2018, 3:02pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957 "2018-03-28T15:02:40Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![janvkn](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@janvkn](https://discuss.elastic.co/u/janvkn)\
**Post date:** [March 28, 2018, 3:02pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957/1 "2018-03-28T15:02:40Z")

</div>

Hello,

I am looking for a solution to do the following: I have a log file where a single log entry can contain multiple key/value pairs that I want to extract. The problem is, I want to extract _all_ occurences in a single entry, not just the first one. (kinda like the "g" option in sed and vim)

For example, a log entry could look like this:

```
2018-03-28 14:23:56 something something user=foo something something more user=bar

```

It's trivial to write the grok filter that matches the first occurrence:

```
grok {
    match => ["logText", "[^A-Z^a-z^0-9]user=(?<user>[A-Za-z0-9]*)" ]
}

```

This would give me a document like this:

```
{
    "user": "foo"
}

```

What I want though, is this:

```
{
    "user": ["foo","bar"]
}

```

Any idea on how to accomplish this?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 28, 2018, 3:10pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957/2 "2018-03-28T15:10:51Z")

</div>

Perhaps surprisingly, a kv filter will do this.

```auto
kv { field_split => " " value_split => "=" }

```

```auto
       "message" => "2018-03-28 14:23:56 something something user=foo something something more user=bar",
          "user" => [
        [0] "foo",
        [1] "bar"
    ]

```

---

<div class="post-metadata">

**Author:** ![janvkn](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@janvkn](https://discuss.elastic.co/u/janvkn)\
**Post date:** [March 28, 2018, 3:17pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957/3 "2018-03-28T15:17:57Z")

</div>

> [@Badger](#):
>
> Perhaps surprisingly, a kv filter will do this.

Interesting, but I'm not looking to extract all possible kv pairs. For instance, a log entry often contains a lot of "uninterestingvariable=something" entries that I do not want to store.

Also, I would like to find something more generic, and not just applicable to key/value pairs. For example, there could be something like IDs in the logfile that have a specific, and easily matched alphanumeric format.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 28, 2018, 3:24pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957/4 "2018-03-28T15:24:19Z")

</div>

Then I would use a ruby filter to split and then iterate over the fields doing a match on each one.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 2, 2018, 7:29pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957/5 "2018-04-02T19:29:31Z")

</div>

> Interesting, but I'm not looking to extract all possible kv pairs. For instance, a log entry often contains a lot of "uninterestingvariable=something" entries that I do not want to store.

The kv filter's `include_keys` option can deal with that.

> Also, I would like to find something more generic, and not just applicable to key/value pairs. For example, there could be something like IDs in the logfile that have a specific, and easily matched alphanumeric format.

Short of a ruby filter I don't think there's a way of doing that with the standard filters.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 30, 2018, 7:29pm UTC](https://discuss.elastic.co/t/grok-multiple-occurences-of-pattern/125957/6 "2018-04-30T19:29:45Z")

</div>

Closed on April 30, 2018, 7:29pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.