If I use, e.g., '/var/logs/*.log' as a prospector path, and only 'a.log' and 'b.log' exist at startup, is a restart required to begin harvesting a new file 'c.log'?
IOW, will the system periodically re-run the search and start new harvesters?
If I use, e.g., '/var/logs/*.log' as a prospector path, and only 'a.log' and 'b.log' exist at startup, is a restart required to begin harvesting a new file 'c.log'?
IOW, will the system periodically re-run the search and start new harvesters?
A restart is not necessary. The glob will be evaluated periodically.
You can define the scan frequency here: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-details.html#_scan_frequency
You can define the scan frequency here: Configure Filebeat | Filebeat Reference [8.11] | Elastic
The documentation isn't very clear. The parameter defines "how often these files should be checked for changes" which I read as "how often to stat the files to see if we should attempt to read from them again" rather than "how often to expand the filename globs".
@magnusbaeck Good point. We will improve it: https://github.com/elastic/filebeat/issues/158 Thanks for the input.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.