# Logstash filter for filtering out specific words from message

**URL:** <https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853>\
**Category:** Logstash\
**Created:** [June 22, 2022, 9:04am UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853 "2022-06-22T09:04:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Muhammed\_Danish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muhammed_danish/32/55579_2.png) [@Muhammed\_Danish](https://discuss.elastic.co/u/Muhammed_Danish)\
**Post date:** [June 22, 2022, 9:04am UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/1 "2022-06-22T09:04:45Z")

</div>

Hello team,

I am looking for a filter to filter out based on the words in a log message. For example,  
Log messages - ` This is a test message from server hosted in AWS` and `This is a test message coming from server hosted in AWS`  
Requirement - New field with value ` test message from server` and `test message coming from server`. that is, filter out the content between words "test" and "server" and put it into a new field.

I tried grok and dissect filter, but neither met my requirements.

many thanks,  
Danish

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 22, 2022, 4:50pm UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/2 "2022-06-22T16:50:01Z")

</div>

> [@Muhammed\_Danish](#):
>
> I tried grok and dissect filter, but neither met my requirements.

What did you try? Both of those filters can be used to do this.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [June 22, 2022, 9:28pm UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/3 "2022-06-22T21:28:59Z")

</div>

Grok pattern: `This is a %{EXTRACT:program} hosted in AWS`

Custom pattern: `EXTRACT (\btest .* server\b)`

Tested in grokdebug

---

<div class="post-metadata">

**Author:** ![Muhammed\_Danish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muhammed_danish/32/55579_2.png) [@Muhammed\_Danish](https://discuss.elastic.co/u/Muhammed_Danish)\
**Post date:** [June 23, 2022, 8:02am UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/4 "2022-06-23T08:02:20Z")

</div>

Thank you for your prompt response, Rios. Could you please explain how to use that grok filter to filter out messages that contain two known words?

```auto
	2022-06-23T07:13:08.091Z	ERROR	controllers.IPpool	TEST-ERROR - IP inconsistency detected	{"namespace": "smoke-test", "name": "ip-out-of-range-test-ingress", "host": "ip-out-of-range-test.kubernetes-test.example.com", "frontend-ip": "0.0.0.0", "error": "Assigned IP outside of pool range."}
github.com/go-logr/zapr.(*zapLogger).Error
	/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128

```

I need to move the content between "TEST-ERROR" and "[go-logr · GitHub](http://github.com/go-logr)" to a new field, regardless of what comes before or after that. This is how the new field should look:

> IP inconsistency detected {"namespace": "smoke-test", "name": "ip-out-of-range-test-ingress", "host": "[ip-out-of-range-test.kubernetes-test.example.com](http://ip-out-of-range-test.kubernetes-test.example.com)", "frontend-ip": "0.0.0.0", "error": "Assigned IP outside of pool range."}

I greatly appreciate your assistance with this.

Many thanks,  
Danish

---

<div class="post-metadata">

**Author:** ![Muhammed\_Danish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muhammed_danish/32/55579_2.png) [@Muhammed\_Danish](https://discuss.elastic.co/u/Muhammed_Danish)\
**Post date:** [June 23, 2022, 8:07am UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/5 "2022-06-23T08:07:28Z")

</div>

Hi Badger,

I couldn't figure out how to use apt dissect or grok filter between two words. Do you have any examples of how this might be used?

Many thanks,  
Danish

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 23, 2022, 3:23pm UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/6 "2022-06-23T15:23:28Z")

</div>

You could use

```
grok { match => { "message" => "TEST-ERROR%{GREEDYDATA:someField}github.com/go-logr" } }

```

or

```
dissect { mapping => { "message" => "%{}TEST-ERROR%{someField}github.com/go-logr%{}"

```

The latter is probably cheaper.

---

<div class="post-metadata">

**Author:** ![Muhammed\_Danish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/muhammed_danish/32/55579_2.png) [@Muhammed\_Danish](https://discuss.elastic.co/u/Muhammed_Danish)\
**Post date:** [June 24, 2022, 3:42pm UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/7 "2022-06-24T15:42:10Z")

</div>

Thank you very much, Badger, that worked!! I greatly appreciate your assistance with this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 22, 2022, 3:42pm UTC](https://discuss.elastic.co/t/logstash-filter-for-filtering-out-specific-words-from-message/307853/8 "2022-07-22T15:42:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.