I am having issues figuring out the syntax for excluding a list of files in a directory. They all come or none come.
Here is the file listing in the directory I am pushing.
total 232676
drwxr-xr-x 6 deploy deploy 4096 Oct 27 07:14 ..
-rw-rw-r-- 1 deploy deploy 10485962 Nov 5 02:40 EnduranceManagementService31-tracer.8.log
-rw-rw-r-- 1 deploy deploy 10485944 Nov 5 06:10 EnduranceManagementService31-tracer.7.log
-rw-rw-r-- 1 deploy deploy 10486260 Nov 5 09:31 EnduranceManagementService31-tracer.6.log
-rw-rw-r-- 1 deploy deploy 10486185 Nov 5 12:11 EnduranceManagementService31-tracer.5.log
-rw-rw-r-- 1 deploy deploy 10486019 Nov 5 13:33 EnduranceManagementService31-tracer.4.log
-rw-rw-r-- 1 deploy deploy 10559538 Nov 5 14:32 EnduranceManagementService31.10.log
-rw-rw-r-- 1 deploy deploy 10560766 Nov 5 14:32 EnduranceManagementService31.9.log
-rw-rw-r-- 1 deploy deploy 10486314 Nov 5 14:51 EnduranceManagementService31-tracer.3.log
-rw-rw-r-- 1 deploy deploy 10486576 Nov 5 15:03 EnduranceManagementService31.8.log
-rw-rw-r-- 1 deploy deploy 10487941 Nov 5 16:30 EnduranceManagementService31-tracer.2.log
-rw-rw-r-- 1 deploy deploy 10485860 Nov 5 16:31 EnduranceManagementService31.7.log
-rw-rw-r-- 1 deploy deploy 10490736 Nov 5 17:35 EnduranceManagementService31.6.log
-rw-rw-r-- 1 deploy deploy 10559632 Nov 5 17:36 EnduranceManagementService31.5.log
-rw-rw-r-- 1 deploy deploy 10559724 Nov 5 17:36 EnduranceManagementService31.4.log
-rw-rw-r-- 1 deploy deploy 10559267 Nov 5 17:36 EnduranceManagementService31.3.log
-rw-rw-r-- 1 deploy deploy 10527715 Nov 5 17:49 EnduranceManagementService31.2.log
-rw-rw-r-- 1 deploy deploy 10486167 Nov 5 18:22 EnduranceManagementService31-tracer.1.log
-rw-rw-r-- 1 deploy deploy 10548899 Nov 5 19:10 EnduranceManagementService31.1.log
drwxr-xr-x 2 deploy deploy 4096 Nov 5 19:10 .
-rw-rw-r-- 1 deploy deploy 1651540 Nov 5 19:34 EnduranceManagementService31-gc.log
-rw-rw-r-- 1 deploy deploy 7954738 Nov 5 19:35 EnduranceManagementService31-tracer.log
-rw-rw-r-- 1 deploy deploy 34027868 Nov 5 19:35 EnduranceManagementService31-access.log
-rw-rw-r-- 1 deploy deploy 5266139 Nov 5 19:35 EnduranceManagementService31.log
my filebeat config file looks like this.
paths:
- /opt/active/logs/*.log
exclude_files: ['-gc\.log$','-tracer.*']
which works fine because I am not getting any of the tracer or gc files.
the issue is I am trying to exclude ANYTHING that has the pattern of *.?.log or *.??.log like you would look up using the ls command - I have tried a few things like.
'*\..\.log'
'.\*\..\.log'
`*.?.log'
i read somewhere that . is seen as any character so I need to escape it if I want the parse to see the .
if I put any of the listings above it stops all logging.
please help this is a bit convoluted for my brain.
I use this configuration on many servers and the filename prefixes change so that has to be like
Anything.[1-9].log or Anything.10.log should be excluded
I tried this looking at some regex documentation- but it blocks everything.
^[[:alnum:]]+.[[:digit:]]+.log$
exclusion line looks like this- nothing is getting through .
exclude_files: ['-gc\.log$','-tracer.*','^[[:alnum:]]+\.[[:digit:]]+\.log$']
is that because the syntax within each of the items below is different? please help