Filebeat maintains the registry and just uploads new logs between filebeat restarts. Is there a way to specify to read few log files from beginning everytime filebeat restarts while maintaining the registry for other files ? Or can we exclude few files from this registry?
Out of curiosity, what is the exact use-case for this? Which log files do you want to be recollected?
Filebeat log input tries to tail files always. In order to force it to recollect a file from the beginning one must delete the entries from the registry while filebeat is stopped. The registry format uses JSON, but from Filebeat POV the exact format is private and it is not recommended to write scripts that operate on the registry.
Alternatively you might consider to configure 2 filebeat instances. For the second instance just delete the registry between restarts.
Thanks Steffen for the quick reply.
Use Case: There are couple of application version files which gets created/updated only during install/update time. We run performance test on the system and run the filebeat only during that period. We would like to get that data for every test period so that it can be shown on the dashboard. The application can be updated or auto-updated anytime so we have no idea what time period to choose to get that data in the kibana dashboard.
If there is no alternate then 2 filebeats should work.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.