# First level filtering with Filebeat

**URL:** <https://discuss.elastic.co/t/first-level-filtering-with-filebeat/92212>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 7, 2017, 6:18am UTC](https://discuss.elastic.co/t/first-level-filtering-with-filebeat/92212 "2017-07-07T06:18:56Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sarfaraz\_Ahmad](https://avatars.discourse-cdn.com/v4/letter/s/67e7ee/32.png) [@Sarfaraz\_Ahmad](https://discuss.elastic.co/u/Sarfaraz_Ahmad)\
**Post date:** [July 7, 2017, 6:18am UTC](https://discuss.elastic.co/t/first-level-filtering-with-filebeat/92212/1 "2017-07-07T06:18:56Z")

</div>

Hi,

Is it possible to use processors and fields from Filebeat modules and write the filtered out events through the output.file module ? I am hoping filebeat should be flexible enough to do this. I don't want to display in Kibana for now.

My filebeat.yml looks like this.

```
filebeat.prospectors:
- input_type: log
  paths:
    - "/root/logstash/working_dir/*"
  processors:
    - drop_event:
        when:
           equals:
             apache2.access.response_code: 200

filebeat.modules:
    - module: apache2
        access:
         enabled: true
         vars.path: ["/root/logstash/working_dir/*"]
    output.file:
      path: "/root/logstash"
      filename: filebeat_out

```

I run it with

/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml.rpmsave -e -modules=apache2

But this isn't dropping any events where Response code is 200.  
Do I have to configure paths for both prospector and processor ?  
I could not find few fluid examples. Do you think there is room for some small yet explanatory examples like this one in the documentation ?

Regards,  
Sarfaraz

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [July 7, 2017, 9:27am UTC](https://discuss.elastic.co/t/first-level-filtering-with-filebeat/92212/2 "2017-07-07T09:27:03Z")

</div>

Hi @Sarfaraz_Ahmad,

Fields processing happens later in the chain (in logstash or ingest node), if you want to filter those you will need to use a processor matching the string, take [https://www.elastic.co/guide/en/beats/filebeat/master/filtering-and-enhancing-data.html#drop-event-example](https://www.elastic.co/guide/en/beats/filebeat/master/filtering-and-enhancing-data.html#drop-event-example) as an example, using `regexp` matching

Best regards

---

<div class="post-metadata">

**Author:** ![Sarfaraz\_Ahmad](https://avatars.discourse-cdn.com/v4/letter/s/67e7ee/32.png) [@Sarfaraz\_Ahmad](https://discuss.elastic.co/u/Sarfaraz_Ahmad)\
**Post date:** [July 7, 2017, 10:15am UTC](https://discuss.elastic.co/t/first-level-filtering-with-filebeat/92212/3 "2017-07-07T10:15:17Z")

</div>

I will try to be more precise,

What i understand from the documentation is that I can use processors instead of using regex with prospectors to filter events out.  
For that filtering to happen do I need ingest node feature from elasticsearch ? or say grok filter from logstash ?

Quote from the page "  
You define processors in the filebeat.yml file to filter and enhance the data _ **before sending events to the configured output** _." suggests that Filebeat should be able to use these modules and filter them out before sending to ingest node feature or logstash ?

I need to filter out DNS query logs for some specific domains. It would be intensive to ship out the entire DNS query logging to logstash/elasticsearch for filtering. I want to do it Filebeat and not write complex regular expressions for prospectors.  
I would rather just morph apache2/nginx or anyother filebeat module to achieve this.

But is this even possible in the first place? Or are the grok expressions that I see in Filebeat modules  
(here [https://github.com/elastic/beats/blob/master/filebeat/module/apache2/access/ingest/default.json](https://github.com/elastic/beats/blob/master/filebeat/module/apache2/access/ingest/default.json)) intended for ingest node feature of elasticsearch ? and even with modules installed, all the logs will be shipped over to elasticsearch and elasticsearch/ingest does the filtering there ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 4, 2017, 10:15am UTC](https://discuss.elastic.co/t/first-level-filtering-with-filebeat/92212/4 "2017-08-04T10:15:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.