I’ve encountered an interesting situation with Filebeat deployed as a daemon set in Openshift (very similar to Filebeat missing end of logs for k8s pods).
So, we have one dedicated pod with Filebeat (configured with k8s autodiscovery) tailing logs from the containers running in other pods inside the same node. When one of those pods crashes too fast (most often due to an OOM), Filebeat somehow “misses” the very last line of the container log. That last line looks like
Terminating due to java.lang.OutOfMemoryError: Java heap space and could be very useful to detect via an alert.
Has anyone an idea if this issue is known or if there’s a workaround that alleviates this behavior? The sidecar option mentioned in the link I shared above is unfortunately not an option for us.