# Exclude file pattern not working correctly

**URL:** <https://discuss.elastic.co/t/exclude-file-pattern-not-working-correctly/145847>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 24, 2018, 6:49am UTC](https://discuss.elastic.co/t/exclude-file-pattern-not-working-correctly/145847 "2018-08-24T06:49:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![brjisc](https://avatars.discourse-cdn.com/v4/letter/b/6bbea6/32.png) [@brjisc](https://discuss.elastic.co/u/brjisc)\
**Post date:** [August 24, 2018, 6:49am UTC](https://discuss.elastic.co/t/exclude-file-pattern-not-working-correctly/145847/1 "2018-08-24T06:49:10Z")

</div>

Hi,  
With this configuration:

filebeat.inputs:

- type: log

Creating these files in the /tmp/cssd directory:

touch error-foo.log  
touch foo-error.log  
touch foo-error  
touch foo.log.1  
touch foo.log.gz  
touch foo.log

touch error-bar.log  
touch bar-error.log  
touch bar-error  
touch bar.log.1  
touch bar.log.gz  
touch bar.log

Gives the following log output:

2018-08-24T07:31:53.659+0100 INFO registrar/registrar.go:124 States Loaded from registrar: 3  
2018-08-24T07:31:53.659+0100 WARN beater/filebeat.go:354 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.  
2018-08-24T07:31:53.659+0100 INFO crawler/crawler.go:48 Loading Inputs: 1  
2018-08-24T07:31:53.661+0100 INFO log/input.go:118 Configured paths: [/tmp/cssd/\*]  
2018-08-24T07:31:53.661+0100 INFO input/input.go:88 Starting input of type: log; ID: 1762196086849029994  
2018-08-24T07:31:53.661+0100 INFO crawler/crawler.go:82 Loading and starting Inputs completed. Enabled inputs: 1  
2018-08-24T07:31:53.661+0100 INFO cfgfile/reload.go:122 Config reloader started  
2018-08-24T07:31:53.661+0100 INFO cfgfile/reload.go:214 Loading of config files completed.  
2018-08-24T07:32:03.662+0100 INFO log/harvester.go:228 Harvester started for file: /tmp/cssd/bar.log  
2018-08-24T07:32:03.662+0100 INFO log/harvester.go:228 Harvester started for file: /tmp/cssd/foo.log

I cannot work out why Filebeat is starting a Harvester for the bar.log

Ben

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [August 24, 2018, 5:50pm UTC](https://discuss.elastic.co/t/exclude-file-pattern-not-working-correctly/145847/2 "2018-08-24T17:50:06Z")

</div>

Hello @brjisc, The **exclude\_files** options need to operate on the full path and not only of the filename. The regular expressions that you have defined above will never match a complete path.

```auto
exclude_files: ['^error.+$','^.+error$','^bar.+','^.+error.+$','^.+.log.1','.*.gz']

```

You are matching `^bar.+` which would match a string beginning with the world `bar` but not the following string `/tmp/cssd/bar.log`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 21, 2018, 5:50pm UTC](https://discuss.elastic.co/t/exclude-file-pattern-not-working-correctly/145847/3 "2018-09-21T17:50:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.