# Avoiding data loss with filebeat in a K8S environment

**URL:** <https://discuss.elastic.co/t/avoiding-data-loss-with-filebeat-in-a-k8s-environment/318324>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [November 7, 2022, 11:18am UTC](https://discuss.elastic.co/t/avoiding-data-loss-with-filebeat-in-a-k8s-environment/318324 "2022-11-07T11:18:07Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ganapati\_Basimsetti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ganapati_basimsetti/32/112952_2.png) [@Ganapati\_Basimsetti](https://discuss.elastic.co/u/Ganapati_Basimsetti)\
**Post date:** [November 7, 2022, 11:18am UTC](https://discuss.elastic.co/t/avoiding-data-loss-with-filebeat-in-a-k8s-environment/318324/1 "2022-11-07T11:18:07Z")

</div>

Hello There,

We are in the process of setting up filebeat in the K8S environment running as a DaemonSet. We are evaluating the possibilities of data loss - logs generated by the application are not uploaded to Logstash and are no longer available on the host VM.

Filebeat is configured to scan `/var/lib/docker/containers/` for logs. Unless we keep the check\_interval very low - 1s, there is a possibility that a container can be GCed by K8S before filebeat can collect last remaining logs. I have looked at the various options (close.\*) in the filbeat docs but couldn't find anything.

Is there a way to coordinate between the K8S API servers or the Container runtime and filebeat to make sure the GC happens only after the logs are collected by the logstash?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 5, 2022, 1:19pm UTC](https://discuss.elastic.co/t/avoiding-data-loss-with-filebeat-in-a-k8s-environment/318324/2 "2022-12-05T13:19:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
