Filebeat Harvester

Elastic Stack Beats
1

Filebeat is not harvesting all the log file details in prospectors.

Here is my prospectors:

filebeat.prospectors:

  • type: log

    enabled: true

    paths:

    • /var/log/*.log
    • /var/log/secure
    • /var/log/messages

However Filebeat is only harvesting secure and messages file not the other *.log files. Any help is greatly appreciated

2018-02-06T09:42:37Z INFO Loading and starting Prospectors completed. Enabled prospectors: 1
2018-02-06T09:42:37Z INFO Config reloader started
2018-02-06T09:42:37Z INFO Loading of config files completed.
2018-02-06T09:42:37Z INFO Harvester started for file: /var/log/secure
2018-02-06T09:42:37Z INFO Harvester started for file: /var/log/messages
2018-02-06T09:43:07Z INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30003 beat.memstats.gc_next=5655728 beat.memstats.memory_alloc=2842168 beat.memstats.memory_total=6360272 filebeat.events.added=18 filebeat.events.done=18 filebeat.harvester.open_files=2 filebeat.harvester.running=2 filebeat.harvester.started=2 libbeat.config.module.running=0 libbeat.config.reloads=1 libbeat.output.events.acked=11 libbeat.output.events.batches=2 libbeat.output.events.total=11 libbeat.output.read.bytes=12 libbeat.output.type=logstash libbeat.output.write.bytes=974 libbeat.pipeline.clients=1 libbeat.pipeline.events.active=0 libbeat.pipeline.events.filtered=7 libbeat.pipeline.events.published=11 libbeat.pipeline.events.retry=5 libbeat.pipeline.events.total=18 libbeat.pipeline.queue.acked=11 registrar.states.current=5 registrar.states.update=18 registrar.writes=9

2

Hi @charan.gandra,

Does Filebeat have access to those files? Permissions are respected when harvesting files

Br,

3

Thanks. Actually it is working. I didn't notice that it has already harvested those files.

Thanks,

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.