# How to remove full\_message or message from log under filter

**URL:** https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289
**Category:** Logstash
**Created:** [December 3, 2019, 5:14am UTC](https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289 "2019-12-03T05:14:10Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![tharu85](https://avatars.discourse-cdn.com/v4/letter/t/e95f7d/32.png) [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [December 3, 2019, 5:14am UTC](https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289/1 "2019-12-03T05:14:10Z")

</div>

I need to remove full\_message or message field using logstash filter. Keep both these values increase the disk capacity.

I tried following syntax under logstash filter.

```
filter {
    grok {
		remove_field => ["full_message"]
	}
}

```

Above will not remove none of mentioned filed

Then I tried following syntax:

```
filter {
    mutate {
           remove_field => ["full_message"]
    }
}

```

Above syntax drop all data and nothing sent to elasticsearch ( no data)

So my question is, is it possible to remove full\_message or message fields in a log using logstash filter ?

---

<div class="post-metadata">

### Author: ![raihan](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@raihan](https://discuss.elastic.co/u/raihan)
#### Post date: [December 3, 2019, 5:18am UTC](https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289/2 "2019-12-03T05:18:23Z")

</div>

```
mutate {
                remove_field => [
                        "@timestamp",
                        "@version",
                        "author",
                        "tags"
                ]
                }

```

This works for me, change the names accordingly as per your needs

---

<div class="post-metadata">

### Author: ![tharu85](https://avatars.discourse-cdn.com/v4/letter/t/e95f7d/32.png) [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [December 3, 2019, 5:40am UTC](https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289/3 "2019-12-03T05:40:17Z")

</div>

@raihan

I tried your code before and just now copied and replace according to my filed name as well.

But it doesn't work. Still the full\_message and message fields appeared in the parser log

---

<div class="post-metadata">

### Author: ![raihan](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@raihan](https://discuss.elastic.co/u/raihan)
#### Post date: [December 3, 2019, 5:42am UTC](https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289/4 "2019-12-03T05:42:24Z")

</div>

@tharu85  
Please put the full config, just wanna make sure if you are not renaming the fields.

---

<div class="post-metadata">

### Author: ![tharu85](https://avatars.discourse-cdn.com/v4/letter/t/e95f7d/32.png) [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [December 3, 2019, 5:51am UTC](https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289/5 "2019-12-03T05:51:30Z")

</div>

@raihan

Refer below logstash config file

```
input {
   udp {
	  port => 30000 
      type => "fortigate-syslog"
	  codec => plain {
        charset => "ISO-8859-1"
    }
}

filter {
   if [type] == "fortigate-syslog" 
   {
     mutate {
		gsub => [ 
			"message", "<.*>", ""
		]
	 }
	 grok 
	 {
		match => ["message", "(?<msg>.*)"]
	 }
	   
	kv { source => "msg" }
	   
	mutate {
	    add_field => ["log_timestamp", "%{date} %{time}"]
		rename => ["crlevel" , "severity_level"]
		rename => ["devname" , "devicename"]
		rename => ["devid" , "deviceid"]
		rename => ["srcintf" , "srcinterface"]
		rename => ["dstintf" , "dstinterface"]
		rename => ["proto" , "protocol"]
		rename => ["appcat" , "appcategory"]
		rename => ["inintf" , "srcinterface"]
		rename => ["outintf" , "dstinterface"]
		rename => ["locport" , "srcport"]
		rename => ["remport" , "dstport"]
		rename => ["locip" , "srcip"]
		rename => ["remip" , "dstip"]
		rename => ["logdesc" , "log_description"]
		rename => ["peer_notif" , "peer_notification"]
		rename => ["dir" , "direction"]
		rename => ["type" , "data_type"]
		rename => ["log_type", "type"]
		rename => ["level", "log_level"]
		rename => ["sessionid", "session_id"]
		rename => ["action", "action_type"]
		rename => ["policyid", "policy_id"]
		rename => ["sentbyte", "sent_bytes"]
		rename => ["rcvdbyte", "input_bytes"]
		rename => ["sentpkt", "sent_packets"]
		rename => ["rcvdpkt", "input_packets"]
		
		convert => ["srcport", "integer"]
		convert => ["dstport", "integer"]
		convert => ["protocol", "integer"]
		convert => ["duration", "integer"]
		convert => ["policy_id", "integer"]
		convert => ["session_id", "integer"]
		convert => ["sent_bytes", "integer"]
		convert => ["input_bytes", "integer"]
		convert => ["sent_packets", "integer"]
		convert => ["input_packets", "integer"]
		
		remove_field => ["msg", "full_message"]
	}
			
	date {
		match => ["log_timestamp", "YYYY-MM-dd HH:mm:ss", "YYYY-MM-dd HH:mm:ss.ZZZ"]
		target => "log_timestamp"
	}		   
  }			
}

output {
    gelf {
        host => "192.168.1.100"
        port => 5000
    }
}
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 31, 2019, 5:51am UTC](https://discuss.elastic.co/t/how-to-remove-full-message-or-message-from-log-under-filter/210289/6 "2019-12-31T05:51:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
