# Docker implementation - Filebeats 7.17.2 - drop\_events not working

**URL:** <https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [April 30, 2022, 7:57pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647 "2022-04-30T19:57:00Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![amills157](https://avatars.discourse-cdn.com/v4/letter/a/a87d85/32.png) [@amills157](https://discuss.elastic.co/u/amills157)\
**Post date:** [April 30, 2022, 7:57pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/1 "2022-04-30T19:57:00Z")

</div>

I am trying to setup filebeats to monitor some docker containers - I am using autodiscover, but I don't want to be logging anything from my stack (Elastic, Kibana, Filebeats itself). I have tried to use `drop_events` but it's just not working:

```auto
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      processors:
        - drop_event.when:
            - equals.docker.container.name: kibana
            - contains.docker.image.name: kibana
            - equals.docker.container.name: elasticsearch
            - contains.docker.image.name: elasticsearch
            - equals.docker.container.name: filebeat
            - contains.docker.image.name: filebeat

```

I also tried:

```auto
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      processors:
        - drop_event.when.not:
            - equals.docker.container.name: httpd

```

Same issue.

I've found a couple of old posts which are related such as this one [here](https://github.com/elastic/beats/issues/10393) - But they all seem to have gone un-answered.

Any help would be appreciated!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 1, 2022, 1:39am UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/2 "2022-05-01T01:39:22Z")

</div>

Hi @amills157 Welcome to the community.

First observation If you don't want to keep logs from your ELK stack then why are you using the `not` basically that will drop everything except for the stack logs.

> [@amills157](#):
>
> `drop_event.when.not`

Perhaps I am missing something.

Also if you want that to be an `or` you need use an `or`

I don't believe your syntax is correct it is very specific.

> **[Define processors | Filebeat Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html#conditions)**

Also, the drop only work on fields that already exist in the message, not once it will be created if you're using a module or a pipeline.

The container names or images I suspect should be available for you to use but you need to use the correct field names and values.

What I would do is write a single "positive" logic drop and get that to work.. then add with the `or`

```auto
- drop_event:
    when:
      field name : value

```

---

<div class="post-metadata">

**Author:** ![amills157](https://avatars.discourse-cdn.com/v4/letter/a/a87d85/32.png) [@amills157](https://discuss.elastic.co/u/amills157)\
**Post date:** [May 1, 2022, 10:43am UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/3 "2022-05-01T10:43:15Z")

</div>

Hi @stephenb ,

No apologies you're right the `.not` in the first example is a typo from where I copy pasta'd the `httpd` version of the config. I'll amend that now (the actual config being used didn't have that `not` and I have double checked / tested that this morning and I'm still seeing logs from KIbana etc) .

As far as I can work out though the `httpd` config should be dropping everything but traffic from the httpd container (which it isn't).

The container name in this instance is correct as the container is called `httpd` so I'm not sure I understand your point in regards to the `field name : value` as that should match what I am trying in this instance.

I did previously try:

```auto
- drop_event:
    - when:
        - contains:
            docker.image.name: kibana

```

Same issue though I kept seeing kibana logs appearing

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 1, 2022, 3:35pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/4 "2022-05-01T15:35:02Z")

</div>

> [@amills157](#):
>
> I did previously try:
> 
> ```auto
> - drop_event:
> - when:
> - contains:
> docker.image.name: kibana
> 
> ```

First you keep putting in `-` when they are not needed / not correct.

So here is a sample I will use... this is the JSON message in discover

```auto
    "container": {
      "image": {
        "name": "gcr.io/google-samples/istio/helloserver:v0.0.1"
      },
      "runtime": "containerd",
      "id": "c0b67738739fc5877596a3f478002ba1f282c2266ea33df6f54f90243471a94b"
    },

```

 ![Screen Shot 2022-05-01 at 8.33.47 AM](https://us1.discourse-cdn.com/elastic/original/3X/4/c/4c656a1502bec636cf48e485e2c114d1f4f57aef.png)

Say I want to drop this then my drop will be ... and this works I tested it.  
**NOTE** 2 spaces Indent no extra `-`

```auto
      - drop_event:
          when:
            contains:
              container.image.name: helloserver

```

This also works which I actually like better

```auto
      - drop_event.when.contains.container.image.name: helloserver

```

Perhaps show us the actual field name and values .... and we can help

---

<div class="post-metadata">

**Author:** ![amills157](https://avatars.discourse-cdn.com/v4/letter/a/a87d85/32.png) [@amills157](https://discuss.elastic.co/u/amills157)\
**Post date:** [May 1, 2022, 4:20pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/5 "2022-05-01T16:20:23Z")

</div>

Hi @stephenb,

Apologies for any syntax issues there - The top two / originally posted configs had been modelled on those in this [post](https://discuss.elastic.co/t/processor-drop-event-not-dropping-events/72186/5) which made use of the `-` as mine does.

I have amended to follow your example:

```auto
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      processors:
        - drop_event.when.contains.container.image.name: kibana

```

However as shown here that hasn't worked either (kibana feed following the most recent config edit)

 ![Screenshot at 2022-05-01 17-19-48](https://us1.discourse-cdn.com/elastic/original/3X/c/d/cd4db0ae8aa307ead859b2d78261edd005b5df5a.png)

You can see that the image / container names should be matching (or not matching for the httpd rule) but I'm getting all logs regardless of the config used.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 1, 2022, 4:25pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/6 "2022-05-01T16:25:27Z")

</div>

Please show your entire filebeat conf not just a snippet.

Also try this move the processor to the left 2 spaces to be at the same level of the `- type`

```auto
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
    processors:
      - drop_event.when.contains.container.image.name: kibana

```

Technically processors can go at the top level

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 1, 2022, 4:27pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/7 "2022-05-01T16:27:58Z")

</div>

Note if you use the provided sample here

> **[Run Filebeat on Docker | Filebeat Reference \[8.1\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html#_example_configuration_file)**

The processors are at the Top Level and will act on every message.

```auto
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

processors:
  - add_cloud_metadata: ~

output.elasticsearch:
  hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
  username: '${ELASTICSEARCH_USERNAME:}'
  password: '${ELASTICSEARCH_PASSWORD:}'

```

so your processor would go

```auto
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

processors:
  - add_cloud_metadata: ~
  - drop_event.when.contains.container.image.name: kibana <!--- Here

output.elasticsearch:
  hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
  username: '${ELASTICSEARCH_USERNAME:}'
  password: '${ELASTICSEARCH_PASSWORD:}'

```

---

<div class="post-metadata">

**Author:** ![amills157](https://avatars.discourse-cdn.com/v4/letter/a/a87d85/32.png) [@amills157](https://discuss.elastic.co/u/amills157)\
**Post date:** [May 1, 2022, 4:43pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/8 "2022-05-01T16:43:09Z")

</div>

Hi @stephenb,

Moving the processors to the top level has fixed it. I had initially moved the processors inside the autodiscover after seeing previous posts with that (and no comments to suggest it was wrong). Lesson learnt there and thanks for your help - To help anyone who has this issue the full config used is:

```auto
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

processors:
  - drop_event.when.contains.container.image.name: kibana
  - drop_event.when.contains.container.image.name: elasticsearch
  ...
  - drop_event.when.contains.container.image.name: filebeat
 

output.elasticsearch:
...

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 1, 2022, 4:44pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/9 "2022-05-01T16:44:06Z")

</div>

I think it will also work at the same level of the `- type` ... but I think you had it too deep.  
Good ... glad we could help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 29, 2022, 6:44pm UTC](https://discuss.elastic.co/t/docker-implementation-filebeats-7-17-2-drop-events-not-working/303647/10 "2022-05-29T18:44:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
