# Remove message on filter before inserting in ES

**URL:** <https://discuss.elastic.co/t/remove-message-on-filter-before-inserting-in-es/31776>\
**Category:** Logstash\
**Created:** [October 7, 2015, 1:55pm UTC](https://discuss.elastic.co/t/remove-message-on-filter-before-inserting-in-es/31776 "2015-10-07T13:55:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ashwin2011](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@ashwin2011](https://discuss.elastic.co/u/ashwin2011)\
**Post date:** [October 7, 2015, 1:55pm UTC](https://discuss.elastic.co/t/remove-message-on-filter-before-inserting-in-es/31776/1 "2015-10-07T13:55:10Z")

</div>

Hi,

Here is my actual config and output :

filter {

if [type] == "ufdb" {

```
            grok {
                    match => ["message", "%{TIMESTAMP_ISO8601:logdate}%{SPACE}%{DATA}%{SPACE}%{WORD:status}%{SPACE}%{USERNAME:group}%{SPACE}%{IP:ip}%{SPACE}%{DATA}%{SPACE}%{WORD:category}%{SPACE}%{URI:url} %{WORD:method}"]

                    }

            date {
      match => ["logdate", "ISO8601", "YYYY-MM-dd HH:mm:ss"]
      target => "logdate"
      locale => "en"
    }
            mutate {
                    remove => ['file','offset','host']
  }

```

}

output on ES :

"hits": {  
"total": 8165,  
"max\_score": 1,  
"hits": [  
{  
"\_index": "shit",  
"\_type": "ufdb",  
"\_id": "AVBCidxwvfdc1\_d4SQML",  
"\_score": 1,  
"\_source": {  
"message": "2015-11-30 15:56:06 [2785] PASS level3 10.249.10.91 level3 ads [http://dm.ccmbg.com/28](http://dm.ccmbg.com/28).... GET",  
"@version": "1",  
"@timestamp": "2015-10-07T13:42:20.958Z",  
"type": "ufdb",  
"logdate": "2015-11-30T14:56:06.000Z",  
"status": "PASS",  
"group": "level3",  
"ip": "10.249.10.91",  
"category": "ads",  
"url": "[http://dm.ccmbg.com/28](http://dm.ccmbg.com/28)....",  
"method": "GET"  
}  
},

On my output I want to remove the message field like I have done for "file,host,offset"

When I do a remove or remove\_field, I just don’t get any data in my index.

Is it possible to remove the message ?

Here is the output I want :

```
  "hits": {
  "total": 8165,
  "max_score": 1,
  "hits": [
     {
        "_index": "shit",
        "_type": "ufdb",
        "_id": "AVBCidxwvfdc1_d4SQML",
        "_score": 1,
        "_source": {
           "@version": "1",
           "@timestamp": "2015-10-07T13:42:20.958Z",
           "type": "ufdb",
           "logdate": "2015-11-30T14:56:06.000Z",
           "status": "PASS",
           "group": "level3",
           "ip": "10.249.10.91",
           "category": "ads",
           "url": "http://dm.ccmbg.com/28....",
           "method": "GET"
        }
     },

```

Any ideas ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 7, 2015, 2:07pm UTC](https://discuss.elastic.co/t/remove-message-on-filter-before-inserting-in-es/31776/2 "2015-10-07T14:07:10Z")

</div>

`message` isn't different from any other field and you can remove it as you please. Removal of that field will not stop the remains of the message to be sent to ES.

---

<div class="post-metadata">

**Author:** ![ashwin2011](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@ashwin2011](https://discuss.elastic.co/u/ashwin2011)\
**Post date:** [October 9, 2015, 7:02am UTC](https://discuss.elastic.co/t/remove-message-on-filter-before-inserting-in-es/31776/3 "2015-10-09T07:02:08Z")

</div>

Thanks but there when I put ''message'' in the remove parameter, my filter don't work at all. I guess the grok don't not match the log line.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 9, 2015, 7:26am UTC](https://discuss.elastic.co/t/remove-message-on-filter-before-inserting-in-es/31776/4 "2015-10-09T07:26:25Z")

</div>

It sounds like you're removing the `message` field _before_ the grok filter that attempts to parse that field. Filters are evaluated in order and if you have multiple configuration files in /etc/logstash/conf.d they're read in alphabetical order.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:27am UTC](https://discuss.elastic.co/t/remove-message-on-filter-before-inserting-in-es/31776/5 "2017-07-06T05:27:04Z")

</div>