# Logstash pipelines: "if \[fileset\]\[module\]" sometimes not working properly

**URL:** <https://discuss.elastic.co/t/logstash-pipelines-if-fileset-module-sometimes-not-working-properly/160999>\
**Category:** Logstash\
**Created:** [December 15, 2018, 12:35pm UTC](https://discuss.elastic.co/t/logstash-pipelines-if-fileset-module-sometimes-not-working-properly/160999 "2018-12-15T12:35:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![MartijnS](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martijns/32/38830_2.png) [@MartijnS](https://discuss.elastic.co/u/MartijnS)\
**Post date:** [December 15, 2018, 12:35pm UTC](https://discuss.elastic.co/t/logstash-pipelines-if-fileset-module-sometimes-not-working-properly/160999/1 "2018-12-15T12:35:42Z")

</div>

Hello,

Since a week or so I'm trying Elastic to determine wether or not it's something we can use at work (webhosting company). I've got it all setup (filebeat and metricbeat, Logstash, Elastic and Kibana). Within Logstash I've setup a multiple pipelines. If it's a "filebeat" input, it will check if the fileset.module == apache. If so, it will be send to the apache pipeline. Works like a charm! For today, there are already +- 11.680 documents inserted in corresponding index. This also works for fileset.module == system, etc. Anything else will be send to the filebeat-unsorted-$date index. This also works. However, in the "filebeat-unsorted-$date" index are also some entries that do have the "fileset.module" set to Apache. I don't understand (yet) why they are being processed to the unsorted index instead of the Apache index. This also happens for the system and nginx fileset.modules. The records showing up in the unsorted index, does not appear in the apache index.

Anyone can give me a clue on how this is possible? The related configurations and locations are:  
**pipelines.yml**  
`
/etc/logstash/pipelines.yml
`

**beats.conf** : this is the config file that determines if the input is a metricbeat or filebeat entry. Not yet sure if this is the way to go or to split it up and give each type it's own input, etc.  
`
/etc/logstash/conf.d/beats.yml
`

**filebeat.conf** : handles all filebeat related documents/messages. This is also the configuration that sends Apache documents to the apache pipeline, etc.:  
`
/etc/logstash/conf.d/filebeat.conf
`

**apache\_pipeline.conf** : handles the apache documents, sends them to the according index in Elasticsearch.  
`
/etc/logstash/conf.d/filebeat/apache_pipeline.conf
`  
**filebeat\_pipeline.conf** : handles the "unsorted" documents / documents that couldn't be matched in the filebeat.conf file  
`
/etc/logstash/conf.d/filebeat/filebeat_pipeline.conf
`

I've placed the configurations on [gist.github.com](http://gist.github.com):

> <https://gist.github.com/spiek90/5a8c92396b2897f700f01c21d51d1c79>
>
> There are more than three files. show original

If there is anything else you'd need to know, please let me know! Hope someones see what's missing in this configuration or where I messed up! 😀

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 12, 2019, 12:35pm UTC](https://discuss.elastic.co/t/logstash-pipelines-if-fileset-module-sometimes-not-working-properly/160999/2 "2019-01-12T12:35:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.