# Failed to load BPF probes **URL:** \ **Category:** Elastic Security\ **Tags:** docker\ **Created:** [July 31, 2023, 9:13am UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667 "2023-07-31T09:13:30Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![L1NG](https://avatars.discourse-cdn.com/v4/letter/l/a698b9/32.png) [@L1NG](https://discuss.elastic.co/u/L1NG)\ **Post date:** [July 31, 2023, 9:13am UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/1 "2023-07-31T09:13:30Z")
Hello, does anyone know how to resolve this issue? My kernel version is 6.4.5-1. el7.elrepo.x86\_ 64, the k8s version is v1.23.6, the system is CentOS Linux 7 (Core), and the Docker version is 24.0.5 ```auto state: 3 message: 1 or more components/units in a failed state fleet_state: 2 fleet_message: Connected log_level: info components: - id: cloud_defend/control-default state: state: 2 message: 'Healthy: communicating with pid ''7301''' units: ? unittype: 0 unitid: cloud_defend/control-default-cloud_defend/control-cloud_defend-43d2f730-2b88-11ee-8b45-a5b2911a0150 : state: 4 message: 'config update: 1 failures' payload: errors: - component: bpf-sensor message: Failed to load BPF probes payload: log: failed BPF state ? unittype: 1 unitid: cloud_defend/control-default : state: 4 message: 'config update: 1 failures' payload: errors: - component: bpf-sensor message: Failed to load BPF probes payload: log: failed BPF state features_idx: 1 version_info: name: cloud-defend version: 8.8.2-076abf2 ``` ---
**Author:** ![L1NG](https://avatars.discourse-cdn.com/v4/letter/l/a698b9/32.png) [@L1NG](https://discuss.elastic.co/u/L1NG)\ **Post date:** [July 31, 2023, 9:34am UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/2 "2023-07-31T09:34:05Z")
It seems to be incompatible with my kernel ```auto {"log.level":"error","@timestamp":"2023-07-31T08:58:55.416Z","message":"load bpf progs (Please ensure required Linux capabilities are set in k8s security context. BPF, PERFMON, and SYS_RESOURCE are mandatory): error loading eBPF probes: field KprobeTaskstatsExit: program kprobe__taskstats_exit: apply CO-RE relocations: load kernel spec: no BTF found for kernel version 6.4.1-1.el7.elrepo.x86_64: not supported","component":{"binary":"cloud-defend","dataset":"elastic_agent.cloud_defend","id":"cloud_defend/control-default","type":"cloud_defend/control"},"log":{"source":"cloud_defend/control-default"},"log.logger":"ebpf-sensor","ecs.version":"1.6.0","ecs.version":"1.6.0"} ``` ---
**Author:** ![Norrie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/norrie/32/122814_2.png) [@Norrie](https://discuss.elastic.co/u/Norrie)\ **Post date:** [July 31, 2023, 8:23pm UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/3 "2023-07-31T20:23:22Z")
Hello @L1NG, We appreciate your interest in Defend for Containers and are excited to help you as you deploy the software. Currently, we are testing/supporting the integration solely in AWS and GCP Kubernetes environments. This means most of our testing is completed using Container optimized OS or Amazon Linux. We would love to support your use case; however, the BPF probes require a Linux kernel with BTF enabled. BTF is a BPF feature that allows generic BPF programs to run across multiple kernel compilations. The elrepo kernel distribution does not have BTF enabled. Is it possible for you to test on a distribution that supports BTF? This requires the kernel to be compiled with `CONFIG_DEBUG_INFO_BTF` enabled. Most major distributions have kernels that do support this out of the box. Some additional requirements of the Defend for Containers integration are that it is deployed on a 5.10.50 (or newer) kernel and is running Kubernetes 1.24 or newer. ---
**Author:** ![L1NG](https://avatars.discourse-cdn.com/v4/letter/l/a698b9/32.png) [@L1NG](https://discuss.elastic.co/u/L1NG)\ **Post date:** [August 7, 2023, 2:28am UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/4 "2023-08-07T02:28:23Z")
Hello, I downloaded Linux 5.15.63-1. el7.x86\_ 64 from the Linux community, but it doesn't seem to support BTF either. Do I need to download the kernel source code and recompile it myself? Or can you provide me with a kernel version that supports BTF out of the box. ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/e/0e55e3ddfad90ca29bed2aa31afcb79c81cbbfa6.png) ---
**Author:** ![Norrie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/norrie/32/122814_2.png) [@Norrie](https://discuss.elastic.co/u/Norrie)\ **Post date:** [August 21, 2023, 6:39pm UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/5 "2023-08-21T18:39:15Z")
> [@Norrie](#): > > Currently, we are testing/supporting the integration solely in AWS and GCP Kubernetes environments. This means most of our testing is completed using Container optimized OS or Amazon Linux. Hi L1NG, For CentOS Linux 7, I am unaware of a public kernel that would work or you out of the box. Compiling a kernel with the necessary configuration is also an option; however, I cannot guarantee your success as it is not one of our tested environments. I would recommend testing in one of the environments mentioned above before investing much time in recompiling a kernel. ---
**Author:** ![L1NG](https://avatars.discourse-cdn.com/v4/letter/l/a698b9/32.png) [@L1NG](https://discuss.elastic.co/u/L1NG)\ **Post date:** [August 24, 2023, 1:10am UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/6 "2023-08-24T01:10:25Z")
Hi @Norrie, I have successfully compiled kernel 5.15.127 and enabled CONFIG\_ DEBUG\_ INFO\_ BTF=y, but the error shows that I seem to need other kernel modules or capabilities. What else do I need to enable? ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/f/2f919a3906c3413c6ce044a795b66b25a27feece.png) Here is the error message > {"log.level":"error","@timestamp":"2023-08-24T00:50:15.019Z","message":"load bpf progs (Please ensure required Linux capabilities are set in k8s security context. BPF, PERFMON, and SYS\_RESOURCE are mandatory): error loading eBPF probes: field LsmFileOpen: program lsm\_\_file\_open: attach LSM/LSMMac: find target for file\_open LSM hook in modules: iterate modules: get next BTF ID: operation not permitted","component":{"binary":"cloud-defend","dataset":"elastic\_agent.cloud\_defend","id":"cloud\_defend/control-default","type":"cloud\_defend/control"},"log":{"source":"cloud\_defend/control-default"},"log.logger":"ebpf-sensor","ecs.version":"1.6.0","ecs.version":"1.6.0"} Can you provide the kernel's config file ---
**Author:** ![L1NG](https://avatars.discourse-cdn.com/v4/letter/l/a698b9/32.png) [@L1NG](https://discuss.elastic.co/u/L1NG)\ **Post date:** [August 31, 2023, 3:29am UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/7 "2023-08-31T03:29:57Z")
Hello @Norrie , I am trying to install and deploy in the environment you mentioned. How can I verify if the plugin is effective？I am currently displaying the agent integration plugin as normal ---
**Author:** ![Norrie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/norrie/32/122814_2.png) [@Norrie](https://discuss.elastic.co/u/Norrie)\ **Post date:** [August 31, 2023, 9:00pm UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/8 "2023-08-31T21:00:53Z")
First, you will want to check if the integration produces logs. You can observe this by navigating to the "Discover" view in Kibana and selecting the "logs-cloud\_defend\*" data view. If you see logs, then you know the integration is producing events. ![Screenshot 2023-08-31 at 1.54.55 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3cb775695431fa62cc843473848ed3aba6b3853f.jpeg) Once you have confirmed this, you can navigate to the "Kubernetes dashboard" in the security solution for better data visualization. ![Screenshot 2023-08-31 at 1.59.49 PM](https://us1.discourse-cdn.com/elastic/original/3X/7/0/70b1991a0a26460389617efa7457b6194697aaa2.png) ---
**Author:** ![L1NG](https://avatars.discourse-cdn.com/v4/letter/l/a698b9/32.png) [@L1NG](https://discuss.elastic.co/u/L1NG)\ **Post date:** [September 5, 2023, 6:37am UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/9 "2023-09-05T06:37:19Z")
Hello, did you test using EKS Kubernetes Worker AMI with Amazon Linux2 image? I am using this image but will report an error as follows ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/e/5e0f19e886b064189caa9e6a38fac0da13e814ec.png) environment：ami-08fd36634c00dfab3 amazon/amazon-eks-node-1.22-v20220429 EKS Kubernetes Worker AMI with AmazonLinux2 image, (k8s: 1.22.6, docker: 20.10.13-2.amzn2, containerd: 1.4.13-2.amzn2.0.1) ---
**Author:** ![Norrie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/norrie/32/122814_2.png) [@Norrie](https://discuss.elastic.co/u/Norrie)\ **Post date:** [September 6, 2023, 6:28pm UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/10 "2023-09-06T18:28:10Z")
Hello @L1NG. It looks like you are making progress. We do testing EKS; however, as mentioned above, the minimum version we support is 1.24. 1.22 is no longer supported by AWS, so I recommend upgrading your cluster. Please double-check that you are uncommenting the following lines in the YAML defining your K8s agent manifest. ```auto securityContext: runAsUser: 0 # The following capabilities are needed for 'Defend for containers' integration (cloud-defend) # If you are using this integration, please uncomment these lines before applying. capabilities: add: - BPF # (since Linux 5.8) allows loading of BPF programs, create most map types, load BTF, iterate programs and maps. - PERFMON # (since Linux 5.8) allows attaching of BPF programs used for performance metrics and observability operations. - SYS_RESOURCE # Allow use of special resources or raising of resource limits. Used by 'Defend for Containers' to modify 'rlimit_memlock' ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 4, 2023, 6:28pm UTC](https://discuss.elastic.co/t/failed-to-load-bpf-probes/339667/11 "2023-10-04T18:28:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.