# Drop event processor not working on Filebeat

**URL:** <https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 31, 2023, 7:04pm UTC](https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725 "2023-07-31T19:04:41Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![JeremyP](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@JeremyP](https://discuss.elastic.co/u/JeremyP)\
**Post date:** [July 31, 2023, 7:04pm UTC](https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725/1 "2023-07-31T19:04:41Z")

</div>

Hello,

I'm trying to create a drop\_event processor to only allow elasticsearch audit logs which have a request.name = "AuthenticateRequest". Clearly my process it not working as all events are not matching and everything is being dropped.

```auto
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
  - drop_event:
      when:
        not:
          request.name:"AuthenticateRequest"

```

Here is a sample of the data being logged to disk with the key value pair matching the log entries but they do not log to Elasticsearch.

```auto
{"type":"audit", "timestamp":"2023-07-31T14:58:05,675-0400", "cluster.uuid":"mmsQadMlRySqZ5FVIVZu_w", "node.id":"ItmwGws8Qu-xismk7PwdMA", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"monitor", "user.realm":"native1", "user.roles":["kibana_monitor","elasticsearch_monitor"], "origin.type":"rest", "origin.address":"x.x.x.x:47702", "request.id":"Thfm_mHwTkKA25MHTatgag", "action":"cluster:admin/xpack/security/user/authenticate", "request.name":"AuthenticateRequest", "opaque_id":"cc567b01-8f8b-452c-b251-e41d4a061b84", "trace.id":"d6f7a0878ab69b081e0f9a9848c9e6a3"}
{"type":"audit", "timestamp":"2023-07-31T14:58:05,681-0400", "cluster.uuid":"mmsQadMlRySqZ5FVIVZu_w", "node.id":"ItmwGws8Qu-xismk7PwdMA", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"monitor", "user.realm":"native1", "user.roles":["kibana_monitor","elasticsearch_monitor"], "origin.type":"rest", "origin.address":"x.x.x.x:52116", "request.id":"enO8cVK4SiqsnCP27KsM7Q", "action":"cluster:admin/xpack/security/user/authenticate", "request.name":"AuthenticateRequest", "opaque_id":"ab1e85cc-8dfc-4eb3-8aaa-03ffae7fa30c", "trace.id":"d6ac44251f290be0ec069686a55b2878"}

```

If I change the logic to match a specific key value pair, it does not work either using the Elasticsearch audit logs.

I'd appreciate any help. Thank you!

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [August 1, 2023, 8:45am UTC](https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725/2 "2023-08-01T08:45:41Z")

</div>

Hey @JeremyP,

I think you are missing the comparison function `equals`. Try with this:

```auto
  - drop_event:
      when:
        not:
          equals:
            request.name: "AuthenticateRequest"

```

I wonder in any case why Filebeat was not complaining about invalid condition 🤔

---

<div class="post-metadata">

**Author:** ![JeremyP](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@JeremyP](https://discuss.elastic.co/u/JeremyP)\
**Post date:** [August 1, 2023, 2:42pm UTC](https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725/3 "2023-08-01T14:42:57Z")

</div>

Thank you. I've made the modification and no filebeat events are streaming into Elasticsearch.... it's dropping everything.

```auto
  - drop_event:
      when:
        not:
          equals:
            request.name:"AuthenticateRequest"

```

I'd expect both these logs to show up in the cluster, but they do not. If I remove the drop\_event filter, everything streams as normal.

```auto
{"type":"audit", "timestamp":"2023-08-01T10:41:15,679-0400", "cluster.uuid":"mmsQadMlRySqZ5FVIVZu_w", "node.id":"ItmwGws8Qu-xismk7PwdMA", "event.type":"transport", "event.action":"access_granted
", "authentication.type":"REALM", "user.name":"monitor", "user.realm":"native1", "user.roles":["kibana_monitor","elasticsearch_monitor"], "origin.type":"rest", "origin.address":"x.x.x.x:413
80", "request.id":"OWY3qrrCTlud4MmeMhlNYg", "action":"cluster:admin/xpack/security/user/authenticate", "request.name":"AuthenticateRequest", "opaque_id":"6a799602-ebdb-45a1-9f0f-2ace874753d3", "
trace.id":"18074220133e12038d621a4ba73de94f"}
{"type":"audit", "timestamp":"2023-08-01T10:41:15,682-0400", "cluster.uuid":"mmsQadMlRySqZ5FVIVZu_w", "node.id":"ItmwGws8Qu-xismk7PwdMA", "event.type":"transport", "event.action":"access_granted
", "authentication.type":"REALM", "user.name":"monitor", "user.realm":"native1", "user.roles":["kibana_monitor","elasticsearch_monitor"], "origin.type":"rest", "origin.address":"x.x.x.x:534
38", "request.id":"gpF-1K6dRN66quniH0Lf7Q", "action":"cluster:admin/xpack/security/user/authenticate", "request.name":"AuthenticateRequest", "opaque_id":"885d8053-fb89-4aa7-b8e8-288422a40069", "
trace.id":"87b356c74866e5a1245af6819c4116f7"}

```

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [August 2, 2023, 2:59pm UTC](https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725/4 "2023-08-02T14:59:30Z")

</div>

Are you parsing these logs locally or using ingest pipelines? The `request.name` field should exist locally.

---

<div class="post-metadata">

**Author:** ![JeremyP](https://avatars.discourse-cdn.com/v4/letter/j/43a26b/32.png) [@JeremyP](https://discuss.elastic.co/u/JeremyP)\
**Post date:** [August 2, 2023, 6:01pm UTC](https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725/5 "2023-08-02T18:01:38Z")

</div>

Elastic support assisted and they suggested adding this process prior to the drop\_event. According to them, the data has not been decoded yet by filebeat prior to the drop\_event.

```auto
  - decode_json_fields: 
      fields: ["message"]
      target:

```

This has resolved the issue for me.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 30, 2023, 8:02pm UTC](https://discuss.elastic.co/t/drop-event-processor-not-working-on-filebeat/339725/6 "2023-08-30T20:02:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.