# Add field from JSON -logstash filter

**URL:** <https://discuss.elastic.co/t/add-field-from-json-logstash-filter/84721>\
**Category:** Logstash\
**Created:** [May 5, 2017, 1:26pm UTC](https://discuss.elastic.co/t/add-field-from-json-logstash-filter/84721 "2017-05-05T13:26:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![lherrera](https://avatars.discourse-cdn.com/v4/letter/l/90db22/32.png) [@lherrera](https://discuss.elastic.co/u/lherrera)\
**Post date:** [May 5, 2017, 1:26pm UTC](https://discuss.elastic.co/t/add-field-from-json-logstash-filter/84721/1 "2017-05-05T13:26:34Z")

</div>

Continuing the discussion from [Add field from JSON / logstash filter](https://discuss.elastic.co/t/add-field-from-json-logstash-filter/69364):

Hi there,

I am having difficulty extracting a field from a log message using the json filter, I hope you can help.  
I am using logstash 5.2.1, the logs come from Journalbeat. This is my configuration:

input {  
beats {  
port =\> 5044  
}  
}

filter {  
json {  
source =\> "message"  
skip\_on\_invalid\_json =\> true  
remove\_field =\> ["timestamp"]  
add\_field =\> {  
"env" =\> "%{[\_source][fields][environment]}"  
"product" =\> "%{[\_source][fields][product]}"  
"service" =\> "%{[\_source][fields][service]}"  
"team" =\> "%{[\_source][fields][team]}"  
}  
}  
}

output {  
elasticsearch {  
hosts =\> ["[http://elasticsearchhost:9200](http://elasticsearchhost:9200)"]  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

The incoming log is like:  
{  
"\_index": "journalbeat-2017.05.03",  
"\_type": "journal",  
"\_id": "AVvOb3kfj92j5Xhsa2ux",  
"\_score": null,  
"\_source": {  
"gid": "6007",  
"syslog\_identifier": "npm",  
"pid": "33943",  
"type": "journal",  
"uid": "1054",  
"hostname": "Web01",  
"cmdline": "npm ",  
"exe": "/usr/local/bin/node",  
"boot\_id": "5a9d8d2b5428",  
"@version": "1",  
"beat": {  
"hostname": "Web01",  
"name": "journalbeat",  
"version": "5.1.1"  
},  
"host": "Web01",  
"systemd\_slice": "system.slice",  
"comm": "npm",  
"syslog\_facility": "3",  
"input\_type": "journal",  
"machine\_id": "5dbf4ddf84a14008b3e56a68cd9a02c0",  
"transport": "stdout",  
"message": "{"level":"ERROR","message":"Error: Page /version/12345678 does not exist\n at new NotFoundError (/opt/frontend/src/main/errors.ts:9:5)\n at /opt/frontend/src/main/app.ts:83:8\n at Layer.handle [as handle\_request] (/opt/frontend/node\_modules/express/lib/router/layer.js:95:5)\n at trim\_prefix (/opt/frontend/node\_modules/express/lib/router/index.js:317:13)\n at /opt/frontend/node\_modules/express/lib/router/index.js:284:7\n ","rootRequestId":"","requestId":"","originRequestId":"","responseCode":"","fields":[],"timestamp":"2017-05-03T13:11:03+00:00","type":"nodejs","microservice":"frontend","team":"cmc","environment":"development","hostname":"Web01"}",  
"priority": "6",  
"tags": [  
"application",  
"beats\_input\_codec\_plain\_applied"  
],  
"@timestamp": "2017-05-03T13:11:03.695Z",  
"systemd\_unit": "cmc-citizen-frontend.service",  
"systemd\_cgroup": "/system.slice/frontend.service",  
"selinux\_context": "system\_u:system\_r:init\_t:s0",  
"cap\_effective": "0",  
"fields": {  
"product": "myproduct",  
"environment": "dev",  
"service": "web",  
"team": "myteam"  
}  
},  
..

I need to extract the fields:  
"fields": {  
"product": "myproduct",  
"environment": "dev",  
"service": "web",  
"team": "myteam"  
}

and add them to the output message.

What i get is the actual string like:  
"fields.team": "%{[\_source][fields][team]}",

not the value of the field.team

Thanks in advance  
Laura

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 5, 2017, 2:08pm UTC](https://discuss.elastic.co/t/add-field-from-json-logstash-filter/84721/2 "2017-05-05T14:08:00Z")

</div>

The event payload doesn't include the `_index`, `_type`, `_id`, and `_score` fields. That's part of the ES query metadata. The payload itself is found under `_source`. In other words, use `%{[fields][team]}` instead of `%{[_source][fields][team]}`.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 2, 2017, 2:17pm UTC](https://discuss.elastic.co/t/add-field-from-json-logstash-filter/84721/3 "2017-06-02T14:17:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.