# Issue with remove\_field

**URL:** https://discuss.elastic.co/t/issue-with-remove-field/72978
**Category:** Logstash
**Created:** [January 27, 2017, 6:03am UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978 "2017-01-27T06:03:53Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![blahnik](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@blahnik](https://discuss.elastic.co/u/blahnik)
#### Post date: [January 27, 2017, 6:03am UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978/1 "2017-01-27T06:03:53Z")

</div>

I am having a hard time understanding why my filter to remove some fields is not working.

I grabbed a sample of the JSON from kibana

> {  
> "\_index": "logstash-2017.01.27",  
> "\_type": "json",  
> "\_score": 1,  
> "\_source": {  
> "source": "unknown",  
> "type": "json",  
> "duration": 16.13,  
> "view": 2.68,  
> "@version": "1",  
> "host": "ip-10-0-0-111",  
> "action": "index",  
> "api": false,  
> "user\_agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36",  
> "controller": "marketing",  
> "format": "html",  
> "message": "{"method":"GET","path":"/","format":"html","controller":"marketing","action":"index","status":200,"duration":16.13,"view":2.68,"db":6.11,"route":"marketing#index","parameters":{},"user\_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36","source":"unknown","tags":["request"],"@timestamp":"2017-01-27T05:20:00Z","@version":"1"}",  
> "tags": [  
> "request"  
> ],  
> "@timestamp": "2017-01-27T05:20:00.000Z",  
> "route": "marketing#index",  
> "parameters": {},  
> "db": 6.11,  
> "status": 200  
> },  
> "fields": {  
> "@timestamp": [  
> 1485494400000  
> ]  
> }  
> }

My logstash.conf file looks like this

> filter {  
> json {  
> source =\> "message"  
> }  
> mutate {  
> remove\_field =\> ["[\_source][message][method]" ]  
> }  
> }

I have also tried  
remove\_field =\> ["[method]"]  
remove\_field =\> ["[message][method]"]

Also if it matters that data is coming in from redis using the input redis and codec =\> json.

Thanks in advance.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 27, 2017, 6:29am UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978/2 "2017-01-27T06:29:55Z")

</div>

The `[_source]` part shouldn't be there at all. The reason `[message][method]` and `[method]` don't work is because your event has no such fields. It only has a `message` field that contains a JSON string. You have to parse that JSON payload, e.g. by setting the codec of your redis input to `json`.

---

<div class="post-metadata">

### Author: ![blahnik](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@blahnik](https://discuss.elastic.co/u/blahnik)
#### Post date: [January 27, 2017, 3:14pm UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978/3 "2017-01-27T15:14:33Z")

</div>

Maybe I have the input wrong but I do have codex =\> json as part of the input. Do i need to do something to parse a block of the input?

input {  
redis {  
host =\>...................  
data\_type =\> 'list'  
codec =\> "json"  
}  
}

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 27, 2017, 3:17pm UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978/4 "2017-01-27T15:17:45Z")

</div>

Um, never mind. I read your previous posts too quickly. It does indeed extract stuff from the JSON payload. But there's no `method` field. Doesn't that in fact indicate that your filter is working?

---

<div class="post-metadata">

### Author: ![blahnik](https://avatars.discourse-cdn.com/v4/letter/b/e36b37/32.png) [@blahnik](https://discuss.elastic.co/u/blahnik)
#### Post date: [January 27, 2017, 4:02pm UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978/5 "2017-01-27T16:02:42Z")

</div>

I guess to be more clear I am trying to remove some fields from the message part of the event, which is a hash inside the event. For example I want to remove the [message][method]..which is one of the keys inside the message.

ie this part:

> "message": "{"method":"GET","path":"/","format":"html","controller":"marketing","action":"index","status":200,"duration":16.13,"view":2.68,"db":6.11,"route":"marketing#index","parameters":{},"user\_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36","source":"unknown","tags":["request"],"@timestamp":"2017-01-27T05:20:00Z","@version":"1"}",

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 27, 2017, 6:32pm UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978/6 "2017-01-27T18:32:48Z")

</div>

> I guess to be more clear I am trying to remove some fields from the message part of the event, which is a hash inside the event.

Your `message` field is a string that happens to be JSON. You can't manipulate that JSON object directly, but you can deserialize it into discrete fields that you can play with. That's what you're doing with the json filter.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 24, 2017, 6:32pm UTC](https://discuss.elastic.co/t/issue-with-remove-field/72978/7 "2017-02-24T18:32:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
