I am using filebeats on my elk stack (recently migrated from logstash-forwarder). Filebeats doesn't releases the deleted log files which is a major concern for us.
I have used the following config, still it holds up the deleted files.
filebeat:
prospectors:
-
paths
- /var/log/syslog
input_type: log
fields:
environment:
app:
app_ip:
ignore_older: 1h
force_close_files: true
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: [":5043"]
tls:
# List of root certificates for HTTPS server verifications
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
shipper:
logging:
rotateeverybytes: 10485760 # = 10MB