Problem with Endpoint Security Initiation

Elastic Security Endpoint Security
1

Hello, I'm trying to initialize Security Endpoint on a Linux Ubuntu 20.04.4 LTS. After that I had a 13 problems for example: "Configure File Events Failure enabling file events; current state is disabled". I tried to diagnose the problem and found:

image
image679×672 38.5 KB

  1. ./elastic-agent status
Status: DEGRADED
Message: app endpoint-security--8.4.2-8ff7857f: Protecting with policy {3de02602-0086-4935-99c7-13141468ea06}
Applications:
  * filebeat               (HEALTHY)
                           Running
  * filebeat_monitoring    (HEALTHY)
                           Running
  * metricbeat_monitoring  (HEALTHY)
                           Running
  * endpoint-security      (DEGRADED)
                           Protecting with policy {3de02602-0086-4935-99c7-13141468ea06}
  * metricbeat             (HEALTHY)
                           Running
  1. ./elastic-agent diagnostics
  *  name: endpoint-security      route_key: default
     error: Get "http://unix/": dial unix /opt/Elastic/Agent/data/tmp/default/endpoint-security/endpoint-security.sock: connect: no such file or directory
  1. Logs from: cat endpoint-000000.log | grep error
{"@timestamp":"2022-10-12T07:35:37.084461884Z","agent":{"id":"","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":312,"name":"SystemLib.cpp"}}},"message":"SystemLib.cpp:312 Failed to stat device [/dev/loop3]","process":{"pid":1352223,"thread":{"id":1352223}}}
{"@timestamp":"2022-10-12T07:35:37.08508799Z","agent":{"id":"","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":312,"name":"SystemLib.cpp"}}},"message":"SystemLib.cpp:312 Failed to stat device [/dev/loop3]","process":{"pid":1352223,"thread":{"id":1352223}}}
{"@timestamp":"2022-10-12T07:35:37.090775618Z","agent":{"id":"","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":2876,"name":"Artifacts.cpp"}}},"message":"Artifacts.cpp:2876 Failed to download artifact diagnostic-configuration-v1 - Invalid url","process":{"pid":1352223,"thread":{"id":1352223}}}
{"@timestamp":"2022-10-12T07:35:37.090803332Z","agent":{"id":"","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":646,"name":"Artifacts.cpp"}}},"message":"Artifacts.cpp:646 Artifact diagnostic-configuration-v1 download or verification failed","process":{"pid":1352223,"thread":{"id":1352223}}}
{"@timestamp":"2022-10-12T07:35:37.114191714Z","agent":{"id":"","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":240,"name":"Tux_HostIsolation.cpp"}}},"message":"Tux_HostIsolation.cpp:240 Failed to mount bpf fs at /sys/fs/bpf: error 13","process":{"pid":1352223,"thread":{"id":1352235}}}
{"@timestamp":"2022-10-12T07:35:37.115493153Z","agent":{"id":"","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"info","origin":{"file":{"line":106,"name":"Internal.cpp"}}},"message":"Internal.cpp:106 sqlite3_prepare_v2 failed: rc=1, msg=SQL logic error","process":{"pid":1352223,"thread":{"id":1352223}}}
{"@timestamp":"2022-10-12T07:35:37.163894839Z","agent":{"id":"00000000-0000-0000-0000-000000000000","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":431,"name":"Comms.cpp"}}},"message":"Comms.cpp:431 No valid comms client configured","process":{"pid":1352223,"thread":{"id":1352223}}}
{"@timestamp":"2022-10-12T07:36:00.957014714Z","agent":{"id":"f1431890-4d3a-42dc-b8f1-44693292530a","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":2138,"name":"Config.cpp"}}},"message":"Config.cpp:2138 Initial configuration application failed","process":{"pid":1352223,"thread":{"id":1352305}}}
{"@timestamp":"2022-10-12T07:36:00.957748606Z","agent":{"id":"f1431890-4d3a-42dc-b8f1-44693292530a","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":288,"name":"AgentContext.cpp"}}},"message":"AgentContext.cpp:288 Failed to apply new policy from Agent.","process":{"pid":1352223,"thread":{"id":1352305}}}
{"@timestamp":"2022-10-12T07:40:39.004336797Z","agent":{"id":"f1431890-4d3a-42dc-b8f1-44693292530a","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":2145,"name":"Config.cpp"}}},"message":"Config.cpp:2145 Policy failed to apply and rollback is disabled","process":{"pid":1352223,"thread":{"id":1352232}}}
{"@timestamp":"2022-10-12T07:49:33.659408762Z","agent":{"id":"f1431890-4d3a-42dc-b8f1-44693292530a","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":2145,"name":"Config.cpp"}}},"message":"Config.cpp:2145 Policy failed to apply and rollback is disabled","process":{"pid":1352223,"thread":{"id":1352305}}}
{"@timestamp":"2022-10-12T07:49:33.661166658Z","agent":{"id":"f1431890-4d3a-42dc-b8f1-44693292530a","type":"endpoint"},"ecs":{"version":"1.11.0"},"log":{"level":"error","origin":{"file":{"line":288,"name":"AgentContext.cpp"}}},"message":"AgentContext.cpp:288 Failed to apply new policy from Agent.","process":{"pid":1352223,"thread":{"id":1352305}}}
2

Thanks for all the initial triage work. The line above notes that permission was denied when attempting to mount the bpf filesytem. I wonder if the system is in a restrictive lockdown posture.

What's the output of :

$ cat /sys/kernel/security/lockdown

We support up to the integrity restriction level. confidential disables insertion of kprobes, ebpf, etc.

If lockdown is not in integrity (or lesser) mode we will need to do more digging. In that case, the full endpoint logs could be helpful. Also, enabling debug logging and recapturing logs could also be helpful.

I'm happy to provide a private upload link over DM for log files.

3

Thanks for anserw.

cat /sys/kernel/security/lockdown 
cat: /sys/kernel/security/lockdown: No such file or directory

How can I attach log files?

4

Ok, I attached

5

I noticed the kernel version was 5.13.19-2-pve, are you attempting to install in proxmox container?
Can you access /sys/kernel/debug/tracing/kprobe_events ?

6

Yes, it is proxmox lxc

7

Elastic Endpoint doesn't support installation from within a container. To have visibility into containers, Endpoint must be installed on the container host, i.e. the proxmox server. All containers running on the host will then be monitored.

We do support installing within VMs, though. If transitioning to a VM is not too heavy weight for your use case, Endpoint could be installed to monitor that single VM instance.

8

@Nick_Berlin thanks for your anserw, and explanation

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.