I have filebeat 7.8.1 streaming logs to my elasticsearch via this setup
Application -> Logfile -> Filebeat -> Logstash -> Elasticsearch
The application team is concerned if Filebeat is down, it will cause logs to be missed,
Will Filebeat be able to detect which logs are sent during it's downtime and automatically send them to logstash when it is turned back on?
Filebeat tracks the position of the harvested files, if you stop filebeat and start again later, it will read from the last know position when it was running.
If the log is rotated while filebeat was in a stop state it will not be able to send the events that happened between that time.
But you can always reparse the logs if needed.
I see, thank you very much for the quick response
The application team tried to delete the log file and each time the log file is deleted, in the directory /data/registry/filebeat/log.json keeps growing in size
Would this be a concern if this log.json keeps growing?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.