# Strange behavior in logstash after remove field message

**URL:** <https://discuss.elastic.co/t/strange-behavior-in-logstash-after-remove-field-message/285524>\
**Category:** Logstash\
**Created:** [September 29, 2021, 9:43pm UTC](https://discuss.elastic.co/t/strange-behavior-in-logstash-after-remove-field-message/285524 "2021-09-29T21:43:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![gbeltramelli](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@gbeltramelli](https://discuss.elastic.co/u/gbeltramelli)\
**Post date:** [September 29, 2021, 9:43pm UTC](https://discuss.elastic.co/t/strange-behavior-in-logstash-after-remove-field-message/285524/1 "2021-09-29T21:43:30Z")

</div>

Hi, first of all sorry for my english!  
im using version 7.14 of ELK and i`m having some strange behavior when using  
mutate { remove\_field =\> ["message"] }

I have some logs from a DataPower then I use GROK to make them more usefull, without the mutate logs everything go fine but when I add that line some logs never appear in the Discover in Kibana.

this is my logstash

does anyone have any idea why when doing a remove\_field logs never reach the elastic output? No error in but the way.

```auto
input {
  tcp {
    port => 8089
    tags => [datapower_log]
    codec => multiline {
      pattern => "<14>"
      negate => true
      what => "previous"
    }
 }
}

filter {
	if "INPUT" in [message] {
	    grok {
	     match => { "message" =>"%{NOTSPACE:Codigo}%{SYSLOGTIMESTAMP:fecha} %{GREEDYDATA:nodo}\[%{WORD:ResponseCode}\]\[%{WORD:debug}\]\[%{WORD:level}\]\ wsgw\(%{WORD:WS}\): trans\(%{WORD:trans}\)\[%{WORD:tipo}\]\[%{IP:cliente}\] gtid\(%{WORD:id}\): \nINPUT:(?m)%{GREEDYDATA:INPUT}\nOUTPUT:(?m)%{GREEDYDATA:OUTPUT}" }
	        }
	mutate {
	   remove_field => ["message"]
	}
	}
output {
  if "datapower_log" in [tags] {
    elasticsearch {
      hosts => ['https://xxxxxxxxx:9200']
      index => "dp-log-%{+YYYY.MM.dd}"
      cacert => '/etc/logstash/bps-net-cer-ca.pem'
      ssl => true
      ssl_certificate_verification => false
      user => elastic
      password => xxxxxxxxxxx
    }
   }
  stdout {
    codec => rubydebug { metadata => false }
  }
}

```

---

<div class="post-metadata">

**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\
**Post date:** [October 1, 2021, 6:11pm UTC](https://discuss.elastic.co/t/strange-behavior-in-logstash-after-remove-field-message/285524/2 "2021-10-01T18:11:25Z")

</div>

if the event doesn’t match your grok, then it will be removed because of that mutate plugin

you can remove field when grok pattern matches event by doing

```auto
filter {
  grok { 
     match => { “message”, “<grok_pattern>” }
     remove_field => [“message”] 
  }
}

```

then the field will be removed only if the filter is successful. remove\_field is one of filter common options that can be applied in any filter. more info here

> **[Grok filter plugin | Logstash Reference \[7.15\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#plugins-filters-grok-common-options)**

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 29, 2021, 6:11pm UTC](https://discuss.elastic.co/t/strange-behavior-in-logstash-after-remove-field-message/285524/3 "2021-10-29T18:11:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.