# Drop filter not working

**URL:** <https://discuss.elastic.co/t/drop-filter-not-working/55940>\
**Category:** Logstash\
**Created:** [July 20, 2016, 5:11am UTC](https://discuss.elastic.co/t/drop-filter-not-working/55940 "2016-07-20T05:11:31Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nishant\_goel](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@nishant\_goel](https://discuss.elastic.co/u/nishant_goel)\
**Post date:** [July 20, 2016, 5:11am UTC](https://discuss.elastic.co/t/drop-filter-not-working/55940/1 "2016-07-20T05:11:31Z")

</div>

I have multiple log messages in a file which I am processing using logstash filter plugins. Then, the filtered logs are getting sent to elasticsearch.

There is one field called addID in a log message. I want to drop all the log messages which have a particular addID present. These particular addIDS are present in a ID.yml file.

Scenario: If the addID of a log message matches with any of the addIDs present in the ID.yml file, that log message should be dropped.

Could anyone help me in achieving this?

@magnusbaeck  
PLease help me Sir

Below is my config file.

input {

```
file {
   path => "/Users/jshaw/logs/access_logs.logs
   ignore_older => 0
}

```

}

filter {

```
grok {

    patterns_dir => ["/Users/jshaw/patterns"]
    match => ["message", "%{TIMESTAMP:Timestamp}+{IP:ClientIP}+{URI:Uri}"]

}

kv{
    field_split => "&?"
    include_keys => ["addID"]
    allow_duplicate_values => "false"

}

if [addID] in "/Users/jshaw/addID.yml" {
    drop{}
}

```

}

output {

```
 elasticsearch{
     hosts => ["localhost:9200"]

 } 

```

}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 20, 2016, 8:39am UTC](https://discuss.elastic.co/t/drop-filter-not-working/55940/2 "2016-07-20T08:39:56Z")

</div>

You can't do a lookup like that unfortunately.

---

<div class="post-metadata">

**Author:** ![nishant\_goel](https://avatars.discourse-cdn.com/v4/letter/n/e47774/32.png) [@nishant\_goel](https://discuss.elastic.co/u/nishant_goel)\
**Post date:** [July 20, 2016, 5:40pm UTC](https://discuss.elastic.co/t/drop-filter-not-working/55940/3 "2016-07-20T17:40:14Z")

</div>

Thanks for replying @warkolm

Is there any workaround?

---

<div class="post-metadata">

**Author:** ![jpcarey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpcarey/32/46668_2.png) [@jpcarey](https://discuss.elastic.co/u/jpcarey)\
**Post date:** [July 20, 2016, 9:53pm UTC](https://discuss.elastic.co/t/drop-filter-not-working/55940/4 "2016-07-20T21:53:05Z")

</div>

You might be able to use the translate filter, which you can then conditionally check if the translate was successful (value is equal to new replaced value, added a field, added a tag, etc). Doing a lookup on each incoming message against a large dictionary will be a performance bottleneck.

[https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:47am UTC](https://discuss.elastic.co/t/drop-filter-not-working/55940/5 "2017-07-06T04:47:08Z")

</div>