# Filebeat not dropping the entire events/log

**URL:** <https://discuss.elastic.co/t/filebeat-not-dropping-the-entire-events-log/246875>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [August 30, 2020, 10:47am UTC](https://discuss.elastic.co/t/filebeat-not-dropping-the-entire-events-log/246875 "2020-08-30T10:47:50Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kumar\_Shubham](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kumar_shubham/32/74791_2.png) [@Kumar\_Shubham](https://discuss.elastic.co/u/Kumar_Shubham)\
**Post date:** [August 30, 2020, 10:47am UTC](https://discuss.elastic.co/t/filebeat-not-dropping-the-entire-events-log/246875/1 "2020-08-30T10:47:51Z")

</div>

I don't want to collect logs from all containers, so I want to discard a few container logs.  
In the below case, I don't want to capture logs from logstash and elastic.

Actual:  
In the logs, I see only the "container" field has been dropped from the JSON body nothing else.

Expected:  
Drop the entire JSON message.

```auto
filebeat.inputs:
  - type: docker
    combine_partial: true
    containers:
      path: "/usr/share/dockerlogs/data"
      stream: "stdout"
      ids:
        - "*"
    exclude_files: ['\.gz$']
    ignore_older: 10m
    processors:
      drop_event.when:
        or:
        - equals:
            docker.container.name: "s_logstash_1"
        - equals:
            docker.container.name: "s_es_1"

```

Output:

"container" field is missing.

3e[36mlogstash\_1 |e[0m {  
e[36mlogstash\_1 |e[0m "host" =\> {  
e[36mlogstash\_1 |e[0m "name" =\> "5e15071a0012"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "@version" =\> "1",  
e[36mlogstash\_1 |e[0m "tags" =\> [  
e[36mlogstash\_1 |e[0m [0] "beats\_input\_codec\_plain\_applied"  
e[36mlogstash\_1 |e[0m],  
e[36mlogstash\_1 |e[0m "@timestamp" =\> 2020-08-30T10:01:53.150Z,  
e[36mlogstash\_1 |e[0m "agent" =\> {  
e[36mlogstash\_1 |e[0m "version" =\> "7.7.1",  
e[36mlogstash\_1 |e[0m "id" =\> "da6fda98-f680-43a6-8f38-fc80d0ff661e",  
e[36mlogstash\_1 |e[0m "type" =\> "filebeat",  
e[36mlogstash\_1 |e[0m "ephemeral\_id" =\> "0120ef2f-2ceb-4a8a-b9aa-328dc7a203ee",  
e[36mlogstash\_1 |e[0m "hostname" =\> "5e15071a0012"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "message" =\> "{"type": "server", "timestamp": "2020-08-30T10:01:53,150Z", "level": "INFO", "component": "o.e.c.m.MetaDataIndexTemplateService", "cluster.name": "elasticsearch", "node.name": "08ebabd41a9d", "message": "adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-7-\*]", "cluster.uuid": "09eHr0q7Say8F-1lWM50Lg", "node.id": "d2R6IIMhSW-cF6w8U00nqA" }",  
e[36mlogstash\_1 |e[0m "input" =\> {  
e[36mlogstash\_1 |e[0m "type" =\> "docker"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "stream" =\> "stdout",  
e[36mlogstash\_1 |e[0m "ecs" =\> {  
e[36mlogstash\_1 |e[0m "version" =\> "1.5.0"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "log" =\> {  
e[36mlogstash\_1 |e[0m "file" =\> {  
e[36mlogstash\_1 |e[0m "path" =\> "/usr/share/dockerlogs/data/08ebabd41a9dbf62295cd207365200ff617a371df5427b0857bcf82298bbed52/08ebabd41a9dbf62295cd207365200ff617a371df5427b0857bcf82298bbed52-json.log"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "offset" =\> 25994  
e[36mlogstash\_1 |e[0m }  
e[36mlogstash\_1 |e[0m }  
e[36mlogstash\_1 |e[0m {  
e[36mlogstash\_1 |e[0m "host" =\> {  
e[36mlogstash\_1 |e[0m "name" =\> "5e15071a0012"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "@version" =\> "1",  
e[36mlogstash\_1 |e[0m "tags" =\> [  
e[36mlogstash\_1 |e[0m [0] "beats\_input\_codec\_plain\_applied"  
e[36mlogstash\_1 |e[0m],  
e[36mlogstash\_1 |e[0m "@timestamp" =\> 2020-08-30T10:01:53.796Z,  
e[36mlogstash\_1 |e[0m "agent" =\> {  
e[36mlogstash\_1 |e[0m "version" =\> "7.7.1",  
e[36mlogstash\_1 |e[0m "id" =\> "da6fda98-f680-43a6-8f38-fc80d0ff661e",  
e[36mlogstash\_1 |e[0m "type" =\> "filebeat",  
e[36mlogstash\_1 |e[0m "ephemeral\_id" =\> "0120ef2f-2ceb-4a8a-b9aa-328dc7a203ee",  
e[36mlogstash\_1 |e[0m "hostname" =\> "5e15071a0012"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "message" =\> "{"type": "server", "timestamp": "2020-08-30T10:01:53,796Z", "level": "INFO", "component": "o.e.l.LicenseService", "cluster.name": "elasticsearch", "node.name": "08ebabd41a9d", "message": "license [ec606c74-b286-4354-be8e-229fc46425af] mode [basic] - valid", "cluster.uuid": "09eHr0q7Say8F-1lWM50Lg", "node.id": "d2R6IIMhSW-cF6w8U00nqA" }",  
e[36mlogstash\_1 |e[0m "input" =\> {  
e[36mlogstash\_1 |e[0m "type" =\> "docker"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "stream" =\> "stdout",  
e[36mlogstash\_1 |e[0m "ecs" =\> {  
e[36mlogstash\_1 |e[0m "version" =\> "1.5.0"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "log" =\> {  
e[36mlogstash\_1 |e[0m "file" =\> {  
e[36mlogstash\_1 |e[0m "path" =\> "/usr/share/dockerlogs/data/08ebabd41a9dbf62295cd207365200ff617a371df5427b0857bcf82298bbed52/08ebabd41a9dbf62295cd207365200ff617a371df5427b0857bcf82298bbed52-json.log"  
e[36mlogstash\_1 |e[0m },  
e[36mlogstash\_1 |e[0m "offset" =\> 26466  
e[36mlogstash\_1 |e[0m }  
e[36mlogstash\_1 |e[0m }

---

<div class="post-metadata">

**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\
**Post date:** [August 31, 2020, 12:42pm UTC](https://discuss.elastic.co/t/filebeat-not-dropping-the-entire-events-log/246875/2 "2020-08-31T12:42:24Z")

</div>

Hi!

This is quite weird! Could you share more info about the version you are using?

Also do you see anything suspicious in the logs regarding the `drop_event` processor? The processor is supposed to just drop events when conditions are met. Could you verify that processor does not fail for some reason?

C.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 28, 2020, 2:42pm UTC](https://discuss.elastic.co/t/filebeat-not-dropping-the-entire-events-log/246875/3 "2020-09-28T14:42:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.