I have learned that filebeat can be configured properly to handle log rotation correctly. However, I am not sure if the following situation I am going to mention is possibile to configure or not.
We want to collect logs(which has rotation) from k8s pods and send them to a cloud file server. The cloud server is an NFS system in which we want all the logs of each pod in the cluster is properly collected with preserver log order.
- I want to have a custom ouput which is able to preserve log order even when log rotation occur (i.e filebeat can somehow detect log roation and inform me with the rotation event so that I can do a rotation on the NFS system before the new log file is created)
- Each line of log should preserved their order when delivered to the custom output
Is it possibile to have a custom output like that?