# Filebeat- syntax for regex file exclusion

**URL:** <https://discuss.elastic.co/t/filebeat-syntax-for-regex-file-exclusion/254452>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 5, 2020, 7:44pm UTC](https://discuss.elastic.co/t/filebeat-syntax-for-regex-file-exclusion/254452 "2020-11-05T19:44:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![txmrlevine](https://avatars.discourse-cdn.com/v4/letter/t/5daacb/32.png) [@txmrlevine](https://discuss.elastic.co/u/txmrlevine)\
**Post date:** [November 5, 2020, 7:44pm UTC](https://discuss.elastic.co/t/filebeat-syntax-for-regex-file-exclusion/254452/1 "2020-11-05T19:44:27Z")

</div>

I am having issues figuring out the syntax for excluding a list of files in a directory. They all come or none come.

Here is the file listing in the directory I am pushing.

```auto
total 232676
drwxr-xr-x 6 deploy deploy 4096 Oct 27 07:14 ..
-rw-rw-r-- 1 deploy deploy 10485962 Nov 5 02:40 EnduranceManagementService31-tracer.8.log
-rw-rw-r-- 1 deploy deploy 10485944 Nov 5 06:10 EnduranceManagementService31-tracer.7.log
-rw-rw-r-- 1 deploy deploy 10486260 Nov 5 09:31 EnduranceManagementService31-tracer.6.log
-rw-rw-r-- 1 deploy deploy 10486185 Nov 5 12:11 EnduranceManagementService31-tracer.5.log
-rw-rw-r-- 1 deploy deploy 10486019 Nov 5 13:33 EnduranceManagementService31-tracer.4.log
-rw-rw-r-- 1 deploy deploy 10559538 Nov 5 14:32 EnduranceManagementService31.10.log
-rw-rw-r-- 1 deploy deploy 10560766 Nov 5 14:32 EnduranceManagementService31.9.log
-rw-rw-r-- 1 deploy deploy 10486314 Nov 5 14:51 EnduranceManagementService31-tracer.3.log
-rw-rw-r-- 1 deploy deploy 10486576 Nov 5 15:03 EnduranceManagementService31.8.log
-rw-rw-r-- 1 deploy deploy 10487941 Nov 5 16:30 EnduranceManagementService31-tracer.2.log
-rw-rw-r-- 1 deploy deploy 10485860 Nov 5 16:31 EnduranceManagementService31.7.log
-rw-rw-r-- 1 deploy deploy 10490736 Nov 5 17:35 EnduranceManagementService31.6.log
-rw-rw-r-- 1 deploy deploy 10559632 Nov 5 17:36 EnduranceManagementService31.5.log
-rw-rw-r-- 1 deploy deploy 10559724 Nov 5 17:36 EnduranceManagementService31.4.log
-rw-rw-r-- 1 deploy deploy 10559267 Nov 5 17:36 EnduranceManagementService31.3.log
-rw-rw-r-- 1 deploy deploy 10527715 Nov 5 17:49 EnduranceManagementService31.2.log
-rw-rw-r-- 1 deploy deploy 10486167 Nov 5 18:22 EnduranceManagementService31-tracer.1.log
-rw-rw-r-- 1 deploy deploy 10548899 Nov 5 19:10 EnduranceManagementService31.1.log
drwxr-xr-x 2 deploy deploy 4096 Nov 5 19:10 .
-rw-rw-r-- 1 deploy deploy 1651540 Nov 5 19:34 EnduranceManagementService31-gc.log
-rw-rw-r-- 1 deploy deploy 7954738 Nov 5 19:35 EnduranceManagementService31-tracer.log
-rw-rw-r-- 1 deploy deploy 34027868 Nov 5 19:35 EnduranceManagementService31-access.log
-rw-rw-r-- 1 deploy deploy 5266139 Nov 5 19:35 EnduranceManagementService31.log

```

my filebeat config file looks like this.

```auto
  paths:
    - /opt/active/logs/*.log
  exclude_files: ['-gc\.log$','-tracer.*']

```

which works fine because I am not getting any of the tracer or gc files.

the issue is I am trying to exclude ANYTHING that has the pattern of \*.?.log or \*.??.log like you would look up using the ls command - I have tried a few things like.

```auto
'*\..\.log'
'.\*\..\.log'
`*.?.log'

```

i read somewhere that . is seen as any character so I need to escape it if I want the parse to see the .

if I put any of the listings above it stops all logging.  
please help this is a bit convoluted for my brain.

I use this configuration on many servers and the filename prefixes change so that has to be like

Anything.[1-9].log or Anything.10.log should be excluded

I tried this looking at some regex documentation- but it blocks everything.

^[[:alnum:]]+.[[:digit:]]+.log$

exclusion line looks like this- nothing is getting through .

```auto
exclude_files: ['-gc\.log$','-tracer.*','^[[:alnum:]]+\.[[:digit:]]+\.log$']

```

is that because the syntax within each of the items below is different? please help

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [November 6, 2020, 4:46pm UTC](https://discuss.elastic.co/t/filebeat-syntax-for-regex-file-exclusion/254452/2 "2020-11-06T16:46:27Z")

</div>

Hey @txmrlevine,

I think you can use this pattern to exclude the rotated files: `\.\d+\.log$`.

But if all the files you are trying to ignore are rotated files, then maybe other option you can try is to use [`ignore_older`](https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-input-log.html#filebeat-input-log-ignore-older).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 4, 2020, 6:46pm UTC](https://discuss.elastic.co/t/filebeat-syntax-for-regex-file-exclusion/254452/3 "2020-12-04T18:46:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
