# Filter don't filter and don't drop events in MongoDB log

**URL:** <https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061>\
**Category:** Logstash\
**Created:** [August 26, 2018, 8:59am UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061 "2018-08-26T08:59:59Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![gabberoid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabberoid/32/33466_2.png) [@gabberoid](https://discuss.elastic.co/u/gabberoid)\
**Post date:** [August 26, 2018, 8:59am UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061/1 "2018-08-26T08:59:59Z")

</div>

Hi. I want to filter all events from MongoDB log except ACCESS component.  
My config file is:

filter {  
if [document\_type] == "windows-mongodb" {  
if [mongodb.log.component] != "ACCESS" {  
drop{}  
}  
grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:mongodb.log.timestamp} %{WORD:mongodb.log.severity} %{WORD:mongodb.log.component} \*\[%{WORD:mongodb.log.context}\] %{GREEDYDATA:mongodb.log.message}" }  
}  
}  
}  
I also tried:

if !( "ACCESS" in [message]) {  
drop{}  
}

And tried this:

if "ACCESS" not in [message]) {  
drop{}  
}

And in filebeat.yml I tried this:

exclude\_lines: ['^!ACCESS']

And it don't work!! I get all components from the log.  
Somebody can help me?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 26, 2018, 4:13pm UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061/2 "2018-08-26T16:13:49Z")

</div>

> [@gabberoid](#):
>
> if [document\_type] == "windows-mongodb" {  
> if [mongodb.log.component] != "ACCESS" {  
> drop{}  
> }  
> grok {  
> match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:mongodb.log.timestamp} %{WORD:mongodb.log.severity} %{WORD:mongodb.log.component} \*[%{WORD:mongodb.log.context}] %{GREEDYDATA:mongodb.log.message}" }  
> }  
> }

The test of mongodb.log.component comes before the grok that creates that field, so the test will never be true.

Also, field names containing . are not supported. They work most of the time, right up until they break things.

---

<div class="post-metadata">

**Author:** ![gabberoid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabberoid/32/33466_2.png) [@gabberoid](https://discuss.elastic.co/u/gabberoid)\
**Post date:** [August 26, 2018, 4:47pm UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061/3 "2018-08-26T16:47:41Z")

</div>

I put it after the grok too and it don't works. Also I changed it before to filed like "message" and it don't work

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 27, 2018, 6:00am UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061/4 "2018-08-27T06:00:00Z")

</div>

Show us an example message that Logstash didn't process correctly. Copy/paste from Kibana's JSON tab or use a `stdout { codec => rubydebug }` output.

---

<div class="post-metadata">

**Author:** ![gabberoid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabberoid/32/33466_2.png) [@gabberoid](https://discuss.elastic.co/u/gabberoid)\
**Post date:** [September 2, 2018, 7:35am UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061/5 "2018-09-02T07:35:08Z")

</div>

The message is:

"message": "2018-09-02T08:28:03.144+0100 I NETWORK [conn3037863] end connection 212.18.253.12:61956 (53 connections now open)"

The conf file for logstash has:

if [document\_type] == "windows-mongodb" {  
grok {  
match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:mongodb.log.timestamp} %{WORD:mongodb.log.severity} %{WORD:mongodb.log.component} \*\[%{WORD:mongodb.log.context}\] %{GREEDYDATA:mongodb.log.message}" }  
}  
if [mongodb.log.component] != "ACCESS" {  
drop{}  
}  
}

In each filebeat.yml I added a new field document\_type.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 3, 2018, 6:03am UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061/6 "2018-09-03T06:03:29Z")

</div>

Please show he full message so we can see if `document_type` really contains "windows-mongodb" and whether there's a `_grokparsefailure` tag.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 1, 2018, 6:03am UTC](https://discuss.elastic.co/t/filter-dont-filter-and-dont-drop-events-in-mongodb-log/146061/7 "2018-10-01T06:03:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.