# Massive filtering in modules? Or in Logstash

**URL:** <https://discuss.elastic.co/t/massive-filtering-in-modules-or-in-logstash/204210>\
**Category:** Beats\
**Tags:** ecs-elastic-common-schema, filebeat\
**Created:** [October 18, 2019, 11:10am UTC](https://discuss.elastic.co/t/massive-filtering-in-modules-or-in-logstash/204210 "2019-10-18T11:10:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![widhalmt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/widhalmt/32/8237_2.png) [@widhalmt](https://discuss.elastic.co/u/widhalmt)\
**Post date:** [October 18, 2019, 11:10am UTC](https://discuss.elastic.co/t/massive-filtering-in-modules-or-in-logstash/204210/1 "2019-10-18T11:10:50Z")

</div>

Hi,

I followed the discussion around [Convert Filebeat icinga.\* to ECS by webmat · Pull Request #9294 · elastic/beats · GitHub](https://github.com/elastic/beats/pull/9294) - a pull request to get the `icinga` module of Filebeat compatible to ECS. There were some changes that are bit confusing to me, which I try to clarify in this thread:

> [@Implementing ECS with custom fields](https://discuss.elastic.co/t/implementing-ecs-with-custom-fields/204184):
>
> Hi, We contributed the icinga module for Filebeat and started developing a Logstash pipeline for parsing Icinga logs: [https://github.com/Icinga/icinga-logstash-pipeline](https://github.com/Icinga/icinga-logstash-pipeline). It seems like there are far too many different rules to implement them into the filebeat module or the ingestion pipeline so we opted for a Logstash pipeline containing all rules. I'm currently trying to line up the parsing when logs come from filebeat by reading the log on the filesystem with logs that where already preproces…

What I wanted to ask here is what you think would be the best to proceed: We started implementing a complete Logstash pipeline with all filters for Icinga 2 logs. I'm sure it's way too much filtering to be implemented in the local `icinga` module of Filebeat but should we think about getting it into an igest pipeline for Elasticsearch? The different syntax makes me hesitant because it was already quite a load of work to get the Logstash pipeline to the point where it is now.

What do you think?

Cheers,  
Thomas

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 15, 2019, 11:10am UTC](https://discuss.elastic.co/t/massive-filtering-in-modules-or-in-logstash/204210/2 "2019-11-15T11:10:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.