I am using filebeat to ingest my log file. I need to handle the case where my application scales down and on of the instance is terminated. In this case, is there any way, I can check if all my log lines have been ingested? I need some kind of flag which says filebeat is up to date (or no backlog pending).
Some people use a script reading the registry file from time to time.
Also see this enhancement request: https://github.com/elastic/beats/issues/7743
Can you please elaborate on the script you mentioned?
The registry file is in JSON. Updates are atomic (using rename). So users sometimes use python (or other scripting language) to parse the JSON. It's one array objects, each containing the file name, inode, device id, read offset and others. You can easily inspect the file using jq.
The format and contents is considered private, though. That is it might change one day. We are aware people read the registry file and we will try not to break it if possible, but you never know.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.