Filebeat doesn't ignore .gz file when harvester file

Elastic Stack Beats
1

Hi there,
I just want read log from specific file, exclude all .gz files and this is my filebeat.yml:

- input_type: log

  paths:
    - /var/log/glusterfs/*
  document_type: glusterfs
  exclude_lines: ['^[[:blank:]]|^[[:digit:]]|^\+|^Final graph:']
  exclude_files: ['\.gz$']

But when i run filebeat and check its log, i saw that filebeat didn't ignore .gz files:

2017-11-03T09:14:31+07:00 INFO filebeat start running.
2017-11-03T09:14:31+07:00 INFO Registry file set to: /home/scloud/filebeat/data/registry
2017-11-03T09:14:31+07:00 INFO Loading registrar data from /home/scloud/filebeat/data/registry
2017-11-03T09:14:31+07:00 INFO States Loaded from registrar: 28
2017-11-03T09:14:31+07:00 INFO Loading Prospectors: 2
2017-11-03T09:14:31+07:00 INFO Start sending events to output
2017-11-03T09:14:31+07:00 INFO Starting Registrar
2017-11-03T09:14:31+07:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017-11-03T09:14:31+07:00 INFO Prospector with previous states loaded: 27
2017-11-03T09:14:31+07:00 WARN DEPRECATED: document_type is deprecated. Use fields instead.
2017-11-03T09:14:31+07:00 INFO Starting prospector of type: log; id: 7031097728770697122
2017-11-03T09:14:31+07:00 INFO Prospector with previous states loaded: 1
2017-11-03T09:14:31+07:00 INFO Starting prospector of type: log; id: 13321976236961072684
2017-11-03T09:14:31+07:00 INFO Loading and starting Prospectors completed. Enabled prospectors: 2
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/cinder/cinder-volume.log
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/var-lib-cinder-gluster-volumes-3f62157ec75b4ed8ee96339c6290fd45.log.5.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/var-lib-cinder-gluster-volumes-3f62157ec75b4ed8ee96339c6290fd45.log.2.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/nfs.log.6.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/glustershd.log.4.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/nfs.log.2.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/var-lib-cinder-gluster-volumes-3f62157ec75b4ed8ee96339c6290fd45.log.4.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/var-lib-cinder-gluster-volumes-3f62157ec75b4ed8ee96339c6290fd45.log.6.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/glustershd.log.5.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/var-lib-cinder-gluster-volumes-3f62157ec75b4ed8ee96339c6290fd45.log.3.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/nfs.log.3.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/nfs.log.5.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/nfs.log.4.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/glustershd.log.3.gz
2017-11-03T09:14:31+07:00 INFO Harvester started for file: /var/log/glusterfs/glustershd.log.2.gz

So, could someone help me to solve this problem?
Thanks.

2

Are these files in the registry file?

Can you share your complete filebeat configuration? To verify indentation being correct and no other prospectors do exist.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.