# Extract fields from JSON

**URL:** <https://discuss.elastic.co/t/extract-fields-from-json/59075>\
**Category:** Logstash\
**Created:** [August 26, 2016, 8:35pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075 "2016-08-26T20:35:05Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![ggp](https://avatars.discourse-cdn.com/v4/letter/g/278dde/32.png) [@ggp](https://discuss.elastic.co/u/ggp)\
**Post date:** [August 26, 2016, 8:35pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/1 "2016-08-26T20:35:05Z")

</div>

I have this below JSON coming from RabbitMQ

> event": {  
> "payloadContext": {  
> "messageProfile": {  
> "domain": "ERP",  
> "process": "process-01",  
> "serviceName": "service-01",  
> "serviceVersion": "4.0.0.RELEASE"  
> },  
> "applicationProfile": {  
> "appName": "app-0",  
> "appUser": "test"  
> },  
> "transactionProfile": {  
> "transactionDateTime": {  
> "value": 1472146765000,  
> "timeZoneCode": null,  
> "daylightSavingTimeIndicator": null  
> },  
> "globalTransactionID": "bb4e273b-c0b6-1378-b2d0-8328971f19d5",  
> "repostFlag": null,  
> "transactionMode": null,  
> "environment": "Test",  
> "event": null  
> },  
> "userArea": null  
> }  
> "emailParams": {  
> "fromAddress": "[xxx@abc.com](mailto:xxx@abc.com)",  
> "toAddress": "[yyy@abc.com](mailto:yyy@abc.com)",  
> "subject": xxxxxxxxx,  
> "template": "xxxxx",  
> "avoidDuplicate": true,  
> "attachmentRequired": true,  
> "ttl": 3600000  
> },  
> "ticketParams": null,  
> "rule": null,  
> "@version": "1",  
> "@timestamp": "2016-08-25T17:39:25.442Z"  
> }

i am looking to extract below fields only and output into elasticsearch

payloadContext/serviceName  
applicationProfile/appName  
transactionProfile/transactionMode  
emailParams/fromAddress  
emailParams/toAddress

any input would be very helpful

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 27, 2016, 9:03am UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/2 "2016-08-27T09:03:48Z")

</div>

What have you tried?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 27, 2016, 9:55am UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/3 "2016-08-27T09:55:17Z")

</div>

Use the mutate filter to copy/move the fields you want to keep into new fields (presumably you want them at the top level of the event rather than as nested fields) then use the prune filter to delete everything but those fields. Or, if the whole message is nested under a single top-level `event` field, you can just delete that top-level field after you've saved the fields you're interested in.

---

<div class="post-metadata">

**Author:** ![ggp](https://avatars.discourse-cdn.com/v4/letter/g/278dde/32.png) [@ggp](https://discuss.elastic.co/u/ggp)\
**Post date:** [August 29, 2016, 12:20pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/5 "2016-08-29T12:20:58Z")

</div>

tried below but didn't help,

filter  
{  
mutate  
{  
remove\_field =\>["[event][applicationProfile][appName]" ]  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 12:44pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/6 "2016-08-29T12:44:03Z")

</div>

We need more details. What did the event look like? What's your configuration? What did you get? What did you expected to get instead?

---

<div class="post-metadata">

**Author:** ![ggp](https://avatars.discourse-cdn.com/v4/letter/g/278dde/32.png) [@ggp](https://discuss.elastic.co/u/ggp)\
**Post date:** [August 29, 2016, 2:30pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/7 "2016-08-29T14:30:34Z")

</div>

please find below config & event input

1. config

> input{  
> rabbitmq{  
> host =\> "xxxx"  
> port =\> "5672"  
> user =\> "xxx"  
> password =\> "xxx"  
> exchange =\> "xxx.EXG"  
> queue =\> "xxx.Q"  
> durable =\> "true"  
> vhost =\> "vhost"  
> threads =\> 5  
> }  
> }  
> filter  
> {  
> mutate  
> {  
> remove\_field =\>["[event][applicationProfile][appName]" ]  
> }  
> }

> output {  
> elasticsearch{  
> hosts =\> ["0.0.0.0"]  
> index =\> "emlticks"  
> document\_type =\> "emlticks"  
> }  
> }

1. event data

> {  
> "event": {  
> "payloadContext": {  
> "messageProfile": {  
> "domain": "ERP",  
> "process": "process-a",  
> "serviceName": "testservice",  
> "serviceVersion": "1.0"  
> },  
> "applicationProfile": {  
> "appName": "testapp",  
> "appUser": "user"  
> },  
> "transactionProfile": {  
> "transactionDateTime": {  
> "value": 1472473590000,  
> "timeZoneCode": null,  
> "daylightSavingTimeIndicator": null  
> },  
> "globalTransactionID": "61ddb532-8d84-87f1-8cac-dec421523ea0",  
> "repostFlag": null,  
> "transactionMode": null,  
> "environment": "Test",  
> "event": null  
> },  
> "userArea": null  
> },  
> "document": {  
> "eventActivity": {  
> "event": "PROCESS",  
> "eventCode": "null",  
> "eventSubCode": null,  
> "step": {  
> "value": null,  
> "languageID": null  
> },  
> "status": {  
> "value": "FAILED",  
> "languageID": null  
> },  
> "summary": null,  
> "detail": {  
> "value": null,  
> "languageID": null  
> },  
> "payload": null  
> "businessIdentifier": "50063000002VDygAAG",  
> "alternateBusinessIdentifier": "04kj00000008OS1AAM",  
> "hostName": "aics360-qas\_1",  
> "threadID": "check.release.task.executor-1"  
> }  
> }  
> },  
> "emailParams": {  
> "fromAddress": "[xxx@xxx.com](mailto:xxx@xxx.com)",  
> "toAddress": "[xxx@xxx.com](mailto:xxx@xxx.com)",  
> "subject": "Test- Attention required for service",  
> "template": "common-email-template",  
> "avoidDuplicate": true,  
> "attachmentRequired": true,  
> "ttl": 3600000  
> },  
> "ticketParams": null,  
> "rule": null,  
> "@version": "1",  
> "@timestamp": "2016-08-29T12:26:31.364Z"  
> }  
> }

from the event i am looking to extra few properties like payloadContext.domain, payloadContext.process, payloadContext.serviceName, emailParams.fromAddress,emailParams.toAddress, applicationProfile.appName etc.,

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 2:33pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/8 "2016-08-29T14:33:55Z")

</div>

There is no `[event][applicationProfile][appName]` field. It's named `[event][payloadContext][applicationProfile][appName]`.

---

<div class="post-metadata">

**Author:** ![ggp](https://avatars.discourse-cdn.com/v4/letter/g/278dde/32.png) [@ggp](https://discuss.elastic.co/u/ggp)\
**Post date:** [August 29, 2016, 2:36pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/9 "2016-08-29T14:36:38Z")

</div>

yeah checking.... if i want to remove the tag applicationProfile should i do like below ?

filter  
{  
mutate  
{  
remove\_tag =\>["[event][payloadContext][applicationProfile]" ]  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 29, 2016, 2:37pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/10 "2016-08-29T14:37:58Z")

</div>

Keep using `remove_field`.

Why don't you try it?

---

<div class="post-metadata">

**Author:** ![ggp](https://avatars.discourse-cdn.com/v4/letter/g/278dde/32.png) [@ggp](https://discuss.elastic.co/u/ggp)\
**Post date:** [August 29, 2016, 2:54pm UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/11 "2016-08-29T14:54:24Z")

</div>

yeah it worked with remove\_field, but tried remove\_tag which didn't help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:41am UTC](https://discuss.elastic.co/t/extract-fields-from-json/59075/12 "2017-07-06T04:41:06Z")

</div>