# Grok filter extracting fields from message

**URL:** <https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298>\
**Category:** Logstash\
**Created:** [June 23, 2020, 4:13pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298 "2020-06-23T16:13:48Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![heinrich](https://avatars.discourse-cdn.com/v4/letter/h/e47c2d/32.png) [@heinrich](https://discuss.elastic.co/u/heinrich)\
**Post date:** [June 23, 2020, 4:13pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/1 "2020-06-23T16:13:48Z")

</div>

Hi guys,

I've never used grok patterns before, after hours of looking into this i'm making very slow progress.

Here's a sample of what i have.

2020-06-23 15:30:02.568 [https-jsse-nio-0.0.0.0-8443-exec-58] INFO c.c.d.c.s.a.UserAuthenticationListener - User logged in: AuthenticationEvent{userName='username', sessionId='-303021888', timeout=1800000, licenseType=CONSUMER, productPermissions='bsg,rdm,dsm,catalog,helpdesk,policymanager,datadictionary,admin', action=LOGIN, remoteHost='10.251.35.112', userAgent='Chrome', userAgentVersion='58.0.3029.110', failureReason=null, timestamp=1592919002568}

I only want to extract the below information

UserAuthenticationListener - User logged in: AuthenticationEvent{userName='username', sessionId='-303021888', timeout=1800000, licenseType=CONSUMER, productPermissions='bsg,rdm,dsm,catalog,helpdesk,policymanager,datadictionary,admin', action=LOGIN, remoteHost='10.251.35.112', userAgent='Chrome', userAgentVersion='58.0.3029.110', failureReason=null, timestamp=1592919002568}

Is there anyone that could possibly be so kind to help me with a filter for this, please?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 23, 2020, 4:57pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/2 "2020-06-23T16:57:24Z")

</div>

I would do that using dissect rather than grok

```
dissect { mapping => { "message" => "%{} %{} [%{}] %{} %{someField}" } }
```

---

<div class="post-metadata">

**Author:** ![heinrich](https://avatars.discourse-cdn.com/v4/letter/h/e47c2d/32.png) [@heinrich](https://discuss.elastic.co/u/heinrich)\
**Post date:** [June 23, 2020, 6:34pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/3 "2020-06-23T18:34:17Z")

</div>

Hi Badger,

Am i using it exactly as you provided? Apologies man i'm extremely new to this.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 23, 2020, 6:42pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/4 "2020-06-23T18:42:48Z")

</div>

Assuming you do not want to extract the date, thread name, and log level then yes, use it exactly as I wrote it. If you want to extract those too then you would use

```
dissect { mapping => { "message" => "%{ts} %{+ts} [%{thread}] %{loglevel} %{someField}" } }
```

---

<div class="post-metadata">

**Author:** ![heinrich](https://avatars.discourse-cdn.com/v4/letter/h/e47c2d/32.png) [@heinrich](https://discuss.elastic.co/u/heinrich)\
**Post date:** [June 23, 2020, 7:13pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/5 "2020-06-23T19:13:53Z")

</div>

![2020-06-23 21_07_10-Discover - Kibana](https://us1.discourse-cdn.com/elastic/original/3X/0/7/07ef017d55c9b69cf2cc868740e4c8751ac40c9d.png)

Hey Badger, here's what i have.

The line you provided, is that supposed to split the fields like loglevel highlighted?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 23, 2020, 7:50pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/6 "2020-06-23T19:50:50Z")

</div>

Yes. If you want to split out fields like action, remoteHost, etc., then use a kv filter.

---

<div class="post-metadata">

**Author:** ![heinrich](https://avatars.discourse-cdn.com/v4/letter/h/e47c2d/32.png) [@heinrich](https://discuss.elastic.co/u/heinrich)\
**Post date:** [June 23, 2020, 7:54pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/7 "2020-06-23T19:54:46Z")

</div>

Is it by anyway possible you could give me one example, like action=login and userName=username

I'm trying to understand how it's formulated, please man i'm desperate to get this working ☹

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 23, 2020, 8:18pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/8 "2020-06-23T20:18:12Z")

</div>

```
    dissect { mapping => { "message" => "%{ts} %{+ts} [%{thread}] %{loglevel} %{someField}{%{[@metadata][kvData]}}%{}" } }
    kv {
        source => "[@metadata][kvData]"
        field_split_pattern => ", "
    }

```

I changed the dissect to pull out the data between AuthenticationEvent{ and } into a separate field. That will result in

```
         "someField" => "c.c.d.c.s.a.UserAuthenticationListener - User logged in: AuthenticationEvent",
          "userName" => "username",
          "loglevel" => "INFO",
            "action" => "LOGIN",
         "userAgent" => "Chrome",
  "userAgentVersion" => "58.0.3029.110",
"productPermissions" => "bsg,rdm,dsm,catalog,helpdesk,policymanager,datadictionary,admin",
       "licenseType" => "CONSUMER",
     "failureReason" => "null",
           "timeout" => "1800000",
        "remoteHost" => "10.251.35.112",
         "timestamp" => "1592919002568",
         "sessionId" => "-303021888",
            "thread" => "https-jsse-nio-0.0.0.0-8443-exec-58",
                "ts" => "2020-06-23 15:30:02.568"
```

---

<div class="post-metadata">

**Author:** ![heinrich](https://avatars.discourse-cdn.com/v4/letter/h/e47c2d/32.png) [@heinrich](https://discuss.elastic.co/u/heinrich)\
**Post date:** [June 23, 2020, 8:21pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/9 "2020-06-23T20:21:41Z")

</div>

YOU ARE A LEGEND! THANK YOU so so much

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 21, 2020, 8:21pm UTC](https://discuss.elastic.co/t/grok-filter-extracting-fields-from-message/238298/10 "2020-07-21T20:21:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.