Hello, on kube we currently leverage two iterations of the file_integrity module. First, we monitor paths under /hostfs, this has been stable for us. Second, we use the auditbeat.autodiscover with containerd to monitor container rootfs. Initially, this was working well.
Once a large volume of containers and paths was introduced, we started to have stability and memory issues. We have incrementally increased memory but the problem is now growing. We can't get more capacity (it's been increased several times)
We've gathered a good amount of data comparing fanotify over inotify. Fanotify is available in newer kernels and has significantly less memory usage over inotify (at scale). We would like to request that auditbeat push for supporting fanotify where supported. I'm trying to gather what is needed so we can contribute resources. From what I see so far, it looks like auditbeat uses fsnotify. Fsnotify has fanotify support on roadmap.
Example autodiscover: beats/auditbeat-kubernetes.yaml at master · elastic/beats · GitHub
Fsnotify github: