# Extracting some JSON fields from the message

**URL:** <https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479>\
**Category:** Logstash\
**Created:** [February 7, 2022, 1:49pm UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479 "2022-02-07T13:49:30Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![hs\_shaikh](https://avatars.discourse-cdn.com/v4/letter/h/9fc29f/32.png) [@hs\_shaikh](https://discuss.elastic.co/u/hs_shaikh)\
**Post date:** [February 7, 2022, 1:49pm UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/1 "2022-02-07T13:49:30Z")

</div>

Hello there, I want to extract "applicationOwner" and "proxyResponseCode" JSON fields from my message

My message filed look like this:

message": "TID: [-1234] [2022-02-07 18:59:15,667] INFO {org.wso2.am.analytics.publisher.sample.reporter.LogCounterMetric} - Metric Name: apim:response Metric Value: {proxyResponseCode=500, errorType=null, applicationOwner=admin, , apiType=HTTP}"

I tried JSON source but it's not working

Error parsing json {:source=\>"message" \<LogStash::Json::ParserError: Unrecognized token 'TID': was expecting ('true', 'false' or 'null')

---

<div class="post-metadata">

**Author:** ![sholzhauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sholzhauer/32/110282_2.png) [@sholzhauer](https://discuss.elastic.co/u/sholzhauer)\
**Post date:** [February 7, 2022, 2:02pm UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/2 "2022-02-07T14:02:51Z")

</div>

Hi, you will have to use two parsers/processors.  
Below example is in logstash:

```auto
filter {
  grok {
    match => ["TID: \[%{DATA}\] \[%{DATA\] \[%{DATA:timestamp}\] %{WORD:log_lvl} %{GREEDYDATA:json_src}" ]
  }

  json {
    source => "json_src"
  }
}

```

This will first parse the line and take the non json part out, or more accuratly put the json part in its own field. You can then use the json parser on that field.

This example puts all fields at the root btw.

---

<div class="post-metadata">

**Author:** ![hs\_shaikh](https://avatars.discourse-cdn.com/v4/letter/h/9fc29f/32.png) [@hs\_shaikh](https://discuss.elastic.co/u/hs_shaikh)\
**Post date:** [February 7, 2022, 3:06pm UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/3 "2022-02-07T15:06:10Z")

</div>

Hi @sholzhauer I tried the above solution but it's giving me grok plugin error, I tried updating it as :-

```auto
grok {
    match =>{ "message" => ["TID: [%{DATA}] [%{DATA}] [%{DATA:timestamp}] %{WORD:log_lvl} %{GREEDYDATA:json_src}" ]
  }
}
  json {
    source => "json_src"
  }

```

But it doesn't change anything

---

<div class="post-metadata">

**Author:** ![sholzhauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sholzhauer/32/110282_2.png) [@sholzhauer](https://discuss.elastic.co/u/sholzhauer)\
**Post date:** [February 7, 2022, 3:10pm UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/4 "2022-02-07T15:10:13Z")

</div>

I put one `\[%{DATA}\]` to much in the pattern, it should be

```auto
TID: \[%{DATA}\] \[%{DATA:timestamp}\] %{WORD:log_lvl} %{GREEDYDATA:json_src}

```

---

<div class="post-metadata">

**Author:** ![hs\_shaikh](https://avatars.discourse-cdn.com/v4/letter/h/9fc29f/32.png) [@hs\_shaikh](https://discuss.elastic.co/u/hs_shaikh)\
**Post date:** [February 8, 2022, 9:42am UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/5 "2022-02-08T09:42:32Z")

</div>

Hi @sholzhauer thanks for the pattern it created a field as

_json\_src": {org.wso2.am.analytics.publisher.sample.reporter.LogCounterMetric} - Metric Name: apim:response Metric Value: {proxyResponseCode=500, errorType=null, applicationOwner=admin, , apiType=HTTP}_

How can I extract **applicationOwner** from that and create a new field?

---

<div class="post-metadata">

**Author:** ![sholzhauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sholzhauer/32/110282_2.png) [@sholzhauer](https://discuss.elastic.co/u/sholzhauer)\
**Post date:** [February 8, 2022, 11:08am UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/6 "2022-02-08T11:08:16Z")

</div>

You do so by using the second part:

```auto
json {
  source => "json_src"
}

```

This will extract the json to the root level. Although i'm not entirely sure if this will correctly parse correctly due to the whitespaces and `=` instead of `:`

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 8, 2022, 5:37pm UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/7 "2022-02-08T17:37:21Z")

</div>

You could try

```
    grok { match => { "message" => "{(?<[@metadata][kvData]>[^}]+)}$" } }
    kv { source => "[@metadata][kvData]" field_split => "," trim_key => " " }

```

which will produce

```
          "apiType" => "HTTP",
"proxyResponseCode" => "500",
        "errorType" => "null",
 "applicationOwner" => "admin"
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 8, 2022, 5:37pm UTC](https://discuss.elastic.co/t/extracting-some-json-fields-from-the-message/296479/8 "2022-03-08T17:37:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.