Filebeat Harvester

[charan.gandra (A G)]

Filebeat is not harvesting all the log file details in prospectors.

Here is my prospectors:

filebeat.prospectors:

• type: log

  enabled: true

  paths:

  • /var/log/*.log
  • /var/log/secure
  • /var/log/messages

However Filebeat is only harvesting secure and messages file not the other *.log files. Any help is greatly appreciated

2018-02-06T09:42:37Z INFO Loading and starting Prospectors completed. Enabled prospectors: 1
2018-02-06T09:42:37Z INFO Config reloader started
2018-02-06T09:42:37Z INFO Loading of config files completed.
2018-02-06T09:42:37Z INFO Harvester started for file: /var/log/secure
2018-02-06T09:42:37Z INFO Harvester started for file: /var/log/messages
2018-02-06T09:43:07Z INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30003 beat.memstats.gc_next=5655728 beat.memstats.memory_alloc=2842168 beat.memstats.memory_total=6360272 filebeat.events.added=18 filebeat.events.done=18 filebeat.harvester.open_files=2 filebeat.harvester.running=2 filebeat.harvester.started=2 libbeat.config.module.running=0 libbeat.config.reloads=1 libbeat.output.events.acked=11 libbeat.output.events.batches=2 libbeat.output.events.total=11 libbeat.output.read.bytes=12 libbeat.output.type=logstash libbeat.output.write.bytes=974 libbeat.pipeline.clients=1 libbeat.pipeline.events.active=0 libbeat.pipeline.events.filtered=7 libbeat.pipeline.events.published=11 libbeat.pipeline.events.retry=5 libbeat.pipeline.events.total=18 libbeat.pipeline.queue.acked=11 registrar.states.current=5 registrar.states.update=18 registrar.writes=9

[exekias (Carlos Pérez Aradros)]

Hi @charan.gandra,

Does Filebeat have access to those files? Permissions are respected when harvesting files

Br,

[charan.gandra (A G)]

Thanks. Actually it is working. I didn't notice that it has already harvested those files.

Thanks,