I have a problem with filebeat that it reads the file while the file is being overwritten by another process. This problem result in loading/reading some junk entries by Filebeat.
My current configuration is :
filebeat.inputs:
-
type: log
enabled: true
paths:- /pa/vertica_backup_qa/otherscripts/orgmapload/scripts/filetoload.txt
exclude_lines: ['^Site,Maker']
fields:
document_type: orga-typescan_frequency: 3600s
- /pa/vertica_backup_qa/otherscripts/orgmapload/scripts/filetoload.txt
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.name: "sys_username_mapping"
setup.template.pattern: "sys_username_map*"
json.overwrite_keys: true
output.logstash:
hosts: ["10.129.212.45:5043"]
logging.level: info
Here same file "filetoload" is getting updated(overwritten) once per day at 1:00 PM and the path is on NFS. So I want that filebeat should read/load the file only if it is not being overwritten by any other process
Please suggest