Trying to run the unified elastic-agent 8.15.0 as ds in kubernetes, and running into the following issue.
┌─ fleet
│ └─ status: (HEALTHY) Connected
└─ elastic-agent
├─ status: (DEGRADED) 1 or more components/units in a failed state
└─ endpoint-default
├─ status: (FAILED) Failed: endpoint service missed 3 check-ins
├─ endpoint-default
│ └─ status: (FAILED) Failed: endpoint service missed 3 check-ins
└─ endpoint-default-c71dbdc5-b510-4546-9888-369c6d4a118d
└─ status: (FAILED) Failed: endpoint service missed 3 check-ins
It's not clear as to why as I don't see anything in the logs, and kibana reports this is the "Elastic Security" integration. Generated diagnostics don't really say anything other than "timeout generating diagnostics" or something along those terms. This same error is occurring across all nodes.
leandrojmp
March 27, 2025, 2:49am
2
Can you provide more context on how exactly are you trying to run it? Which configuration are you using?
Here's the DS manifest
apiVersion: apps/v1
kind: DaemonSet
metadata:
annotations:
deprecated.daemonset.template.generation: "16"
meta.helm.sh/release-name: elastic
meta.helm.sh/release-namespace: elastic-system
creationTimestamp: "2025-03-21T23:06:34Z"
generation: 16
labels:
app: elastic-agent
app.kubernetes.io/instance: elastic
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: deus-ex-elastic-agent
app.kubernetes.io/version: 8.15.0
helm.sh/chart: deus-ex-elastic-agent-8.15.0
name: elastic-deus-ex-elastic-agent
namespace: elastic-system
resourceVersion: "37563124"
uid: 4558dfdf-4c2a-46bc-b837-628cd17250b1
spec:
revisionHistoryLimit: 10
selector:
matchLabels:
app: elastic-agent
template:
metadata:
annotations:
kubectl.kubernetes.io/restartedAt: "2025-03-25T23:36:27Z"
creationTimestamp: null
labels:
app: elastic-agent
app.kubernetes.io/instance: elastic
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: deus-ex-elastic-agent
app.kubernetes.io/version: 8.15.0
helm.sh/chart: deus-ex-elastic-agent-8.15.0
spec:
containers:
- env:
- name: FLEET_ENROLL
value: "1"
- name: FLEET_SERVER_ELASTICSEARCH_INSECURE
value: "true"
- name: FLEET_INSECURE
value: "true"
- name: FLEET_URL
value: < redacted >
- name: FLEET_ENROLLMENT_TOKEN
value: < redacted >
- name: NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: ELASTIC_NETINFO
value: "false"
image: localregistry.test.space:5000/beats/elastic-agent:8.15.0
imagePullPolicy: IfNotPresent
name: elastic-agent
resources:
limits:
memory: 2Gi
requests:
memory: 2Gi
securityContext:
capabilities:
add:
- SYS_ADMIN
privileged: true
runAsUser: 0
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /hostfs/proc
name: proc
readOnly: true
- mountPath: /hostfs/sys/fs/cgroup
name: cgroup
readOnly: true
- mountPath: /var/lib/docker/containers
name: varlibdockercontainers
readOnly: true
- mountPath: /var/log
name: varlog
readOnly: true
- mountPath: /hostfs/etc
name: etc-full
readOnly: true
- mountPath: /hostfs/var/lib
name: var-lib
readOnly: true
- mountPath: /etc/machine-id
name: etc-mid
readOnly: true
- mountPath: /sys/kernel/debug
name: sys-kernel-debug
- mountPath: /usr/share/elastic-agent/state
name: elastic-agent-state
- mountPath: /usr/local/share/ca-certificates/elastic-stack-ca.crt
name: ca
subPath: ca.crt
- mountPath: /usr/local/bin/docker-entrypoint
name: sh
subPath: sh
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true
hostPID: true
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: elastic-deus-ex-elastic-agent
serviceAccountName: elastic-deus-ex-elastic-agent
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
- effect: NoSchedule
key: node-role.kubernetes.io/master
- effect: NoExecute
key: CriticalAddonsOnly
operator: Exists
- effect: NoSchedule
key: cpu
operator: Exists
volumes:
- configMap:
defaultMode: 493
name: elastic-deus-ex-elastic-agent-config
name: sh
- name: ca
secret:
defaultMode: 420
secretName: elastic-deus-ex-elastic-agent-certs
- hostPath:
path: /proc
type: ""
name: proc
- hostPath:
path: /sys/fs/cgroup
type: ""
name: cgroup
- hostPath:
path: /var/lib/docker/containers
type: ""
name: varlibdockercontainers
- hostPath:
path: /var/log
type: ""
name: varlog
- hostPath:
path: /etc
type: ""
name: etc-full
- hostPath:
path: /var/lib
type: ""
name: var-lib
- hostPath:
path: /etc/machine-id
type: File
name: etc-mid
- hostPath:
path: /sys/kernel/debug
type: ""
name: sys-kernel-debug
- hostPath:
path: /var/lib/elastic-agent-managed/elastic-system/state
type: DirectoryOrCreate
name: elastic-agent-state
updateStrategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
The workload nodes are all effectively in private subnets which have their outbound access controlled.
leandrojmp
March 27, 2025, 3:11am
4
It looks like that you have a Elastic Agent policy with the Defend integration, right?
This does not work as mentioned here .
Elastic Defend does not support deployment within an Elastic Agent DaemonSet in Kubernetes.
On Kubernertes you need Defender for Containers, which is part of Container Workload Protection .
This is a paid feature, it requires an Enterprise license.