# Filebeat Nginx module not dropping events

**URL:** <https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 13, 2018, 6:17pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797 "2018-12-13T18:17:11Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![arlen](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@arlen](https://discuss.elastic.co/u/arlen)\
**Post date:** [December 13, 2018, 6:17pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797/1 "2018-12-13T18:17:11Z")

</div>

I'm trying to exclude some events, started out with a more complex processor, but was never once able to make even a simple condition work. drop\_event, with no condition, does what it's supposed to and drops everything. But as soon as a try to apply a condition, it refuses to drop any event at all. Here's the nginx.yml  
module config file:

```
- module: nginx

  # Access logs
  access:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:
    input:
      processors:
      - drop_event:
          when:
            equals:
              nginx.access.remote_ip: '127.0.0.1'

```

which should drop any request coming from the system itself. It doesn't. I've also tried to match nginx.access.url and it fails to drop a single event involving the URL in the rule. It shouldn't be this hard to get it to do what it's supposed to do!

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 21, 2018, 1:31pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797/2 "2018-12-21T13:31:50Z")

</div>

UPDATE: I wrote before the config should be on the module level. This is NOT correct and what you have above is correct.

Any chance you could enable debug logging and share the output. Currently trying this out locally.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 21, 2018, 1:43pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797/3 "2018-12-21T13:43:13Z")

</div>

Can you replace `equals` through `contains` in your processor above?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [December 21, 2018, 1:49pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797/4 "2018-12-21T13:49:53Z")

</div>

I think the problem above is that `nginx.access.remote_ip` is only available after the ingest processor. The processing of the log line happens in Elasticsearch, so filebeat cannot filter based on this field as it never sees it.

For your case you would either have to do this in the ingest pipeline or do a regexp on the filebeat side with `exclude_line`: [https://www.elastic.co/guide/en/beats/filebeat/6.5/filebeat-input-log.html#filebeat-input-log-exclude-lines](https://www.elastic.co/guide/en/beats/filebeat/6.5/filebeat-input-log.html#filebeat-input-log-exclude-lines)

---

<div class="post-metadata">

**Author:** ![arlen](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@arlen](https://discuss.elastic.co/u/arlen)\
**Post date:** [December 28, 2018, 3:54pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797/5 "2018-12-28T15:54:19Z")

</div>

Sorry, things got hectic around here for a bit. I was suspecting something like that was the case, as I'd used the drop\_event processor before, but without using the IP address. My ideal case for this would use both nginx.access.url _and_ remote\_ip to decide when to drop the event (one example: drop all events for "/server-status" coming from from a list of IPs representing the various performance analyzers I use).

Is url something that only gets set by ingest, or would it be available to use?

EDIT: Upon further digging, this can work in the minx module, but as @ruflin notes, the nginx fields aren't available at the time of running, so the only way to do it is to concoct a regex test against "message", i.e.:

```
- drop_event.when.regexp.message: "127.0.0.1.*"

```

in order to drop all events coming from 127.0.0.1

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 25, 2019, 3:54pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module-not-dropping-events/160797/6 "2019-01-25T15:54:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.