# How to extract json out of an event?

**URL:** https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684
**Category:** Logstash
**Created:** [June 5, 2018, 7:31pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684 "2018-06-05T19:31:09Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [June 5, 2018, 7:31pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/1 "2018-06-05T19:31:09Z")

</div>

Hi,

Here is my sample event,

`<6>Jun 5 12:12:45 172.25.29.92 doppler[10]: time="2018-06-05T16:12:46Z" level=info msg="{ "Level": "Info", "Timestamp": "2018-06-05 14:12:45.1195", "Message": "sample message" } " app_id=4456 app_name=dev-app org_id=65739 org_name=Dev timestamp=1528215165150529000`

I want to extract msg field(which is a json) and later want to extract key-value pairs out of msg field. Please suggest me a way.

Here is the logstash config I am using,

```
filter{
      grok {
                                match => {"message" => "%{SYSLOGBASE}\stime\=\"%{NOTSPACE:timest}\"\slevel\=%{NOTSPACE:level}\smsg\=\"\{%{DATA:jsondata}\}%{GREEDYDATA:test}"}
                        }
}

```

Problem with above grok would be, I will end up not having closing curly bracket }.

Please suggest me a better way to extract json data.

---

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [June 8, 2018, 2:03pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/2 "2018-06-08T14:03:32Z")

</div>

Any help here is highly appreciated 🙂

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 8, 2018, 3:02pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/3 "2018-06-08T15:02:44Z")

</div>

If you only problem with the jsondata field is that it is missing a curly brace, then one option is to add one.

```
mutate { gsub => ["jsondata", "$", "}'] }
```

---

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [June 8, 2018, 3:38pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/4 "2018-06-08T15:38:02Z")

</div>

Thanks Badger. Can you please explain what is `$` here?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 8, 2018, 4:39pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/5 "2018-06-08T16:39:15Z")

</div>

It is a regular expression that matches the end of the line. So this says to change the end of the line to be }.

---

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [June 13, 2018, 8:25pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/6 "2018-06-13T20:25:18Z")

</div>

Thanks Badger, it works fine.

I have another issue - one of the json fields is named message and also logstash by default gives message field. I want to rename josn field "message" to something like "json\_message". If I use mutate filter, I think it would replace both the fields. Is there any way where I can rename fields only for json fields?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 13, 2018, 9:19pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/7 "2018-06-13T21:19:58Z")

</div>

If you have a field called "message" and you parse the JSON, by default it will overwrite the original "message". If you set target on the JSON filter than you will have the "message" of the JSON inside someField, and then you can rename that using something like

```
mutate { rename => { "[someField][message]" => "[somefield][someOtherName]" } }
```

---

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [June 14, 2018, 4:18pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/8 "2018-06-14T16:18:33Z")

</div>

Thanks Badger.

My Json object may have several fields depending on the event. It may have a least of 3 fields or it may even have up to 20 fields. If I use a target field named jsondoc, all the fields will be inside jsondoc. But I want all the fields to be independent instead in jsondoc. Is there a way?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 14, 2018, 4:28pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/9 "2018-06-14T16:28:27Z")

</div>

Yes, do not set a target, and the fields will be at the root level. In that case, if the JSON includes a field called message it will overwrite the message field. If you really want to retain the message field then mutate+rename message to saved\_message, parse saved\_message with a json filter, mutate+rename message to json\_message, mutate+rename saved\_message back to message..

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 12, 2018, 4:38pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684/10 "2018-07-12T16:38:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
