# Pattern filter in filebeat or ingest node?

**URL:** <https://discuss.elastic.co/t/pattern-filter-in-filebeat-or-ingest-node/197374>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 29, 2019, 3:23pm UTC](https://discuss.elastic.co/t/pattern-filter-in-filebeat-or-ingest-node/197374 "2019-08-29T15:23:35Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rverchere](https://avatars.discourse-cdn.com/v4/letter/r/dbc845/32.png) [@rverchere](https://discuss.elastic.co/u/rverchere)\
**Post date:** [August 29, 2019, 3:23pm UTC](https://discuss.elastic.co/t/pattern-filter-in-filebeat-or-ingest-node/197374/1 "2019-08-29T15:23:35Z")

</div>

Hello,

I have a setup with a filebeat agent that sends messages to an elastic cluster.  
I need to filter messages that goes to the cluster, and I have 2 options:

- Using pattern filters on filebeat
- Using filtering pipelines on ingest node

I have a lot of data (approx 500GB per day, and only 1% will be stored in elastic nodes).  
Is there any recommendations on what is the most confortable solution?

Thanks!

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [August 29, 2019, 8:06pm UTC](https://discuss.elastic.co/t/pattern-filter-in-filebeat-or-ingest-node/197374/2 "2019-08-29T20:06:49Z")

</div>

Hi @rverchere and welcome 🙂

> I have a lot of data (approx 500GB per day, and only 1% will be stored in elastic nodes).  
> Is there any recommendations on what is the most confortable solution?

It depends, but quite probably the best option is to filter them out already in filebeat, this way you avoid the network traffic caused by these messages that you are going to drop in any case.

To drop events from filebeat, you can use the [`drop_events` processor](https://www.elastic.co/guide/en/beats/filebeat/7.3/drop-event.html).

---

<div class="post-metadata">

**Author:** ![rverchere](https://avatars.discourse-cdn.com/v4/letter/r/dbc845/32.png) [@rverchere](https://discuss.elastic.co/u/rverchere)\
**Post date:** [September 2, 2019, 8:53pm UTC](https://discuss.elastic.co/t/pattern-filter-in-filebeat-or-ingest-node/197374/3 "2019-09-02T20:53:30Z")

</div>

Hey, thanks for your feedback!

I think I will go for your solution, and if I have time or find something that does not fit my needs, I will go with pipelines on ingest nodes.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 30, 2019, 8:53pm UTC](https://discuss.elastic.co/t/pattern-filter-in-filebeat-or-ingest-node/197374/4 "2019-09-30T20:53:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.