# Unhealthy EDOT/OTEL Collector in Agent 9.4.x **URL:** \ **Category:** Elastic Agent\ **Tags:** metricbeat\ **Created:** [July 31, 2026, 4:58pm UTC](https://discuss.elastic.co/t/unhealthy-edot-otel-collector-in-agent-9-4-x/388928 "2026-07-31T16:58:10Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![DavidA](https://avatars.discourse-cdn.com/v4/letter/d/ba8739/32.png) [@DavidA](https://discuss.elastic.co/u/DavidA)\ **Post date:** [July 31, 2026, 4:58pm UTC](https://discuss.elastic.co/t/unhealthy-edot-otel-collector-in-agent-9-4-x/388928/1 "2026-07-31T16:58:11Z")
Environment: - Elastic Agent / EDOT: 9.4.4 (build baed89504b1a1c0a7642d60dc0b6c1b0eb34f100, elastic-otel-collector service.version 9.4.4) - OS: Ubuntu 24.04 (HP thin client) - Fleet-managed, policy "Thin Client" - Agent runtime config: agent.internal.runtime.metricbeat.default: otel # metricbeat hosted inside the OTEL collector agent.monitoring.\_runtime\_experimental: otel agent.grpc.port: 6789 # agent \<-\> collector mgmt channel Symptom: After the reimage + upgrade to 9.4.4, the agent reports DEGRADED (state 3) with: collector: error: failed to connect to collector status: 3 timestamp: "2026-07-28T12:34:44Z" ... fleet\_message: Connected # Fleet link is fine fleet\_state: 2 message: 1 or more components/units in a degraded state state: 3 # agent overall DEGRADED The two components that never come up are exactly the two system/metrics-type inputs that run metricbeat inside the EDOT collector (the metricbeatreceiver): - system/metrics-default \> metricbeatreceiver/\_agent-component/system/metrics-default \> elasticsearch/\_agent-component/default - system/metrics-monitoring \> metricbeatreceiver/\_agent-component/system/metrics-monitoring \> elasticsearch/\_agent-component/monitoring Every other input type (log-default, audit/auditd-default, audit/file\_integrity-default, synthetics/tcp-default, osquery-default, endpoint, beat/metrics-monitoring, http/metrics-monitoring, filestream-monitoring) checks in Healthy. What I've verified (evidence) 1. Policy vs. running components. Comparing the agent's expected vs. actual component list, the only two missing are the two system/metrics inputs: Expected (11): log-default, system/metrics-default, audit/auditd-default, audit/file\_integrity-default, synthetics/tcp-default, endpoint, osquery-default, http/metrics-monitoring, system/metrics-monitoring, filestream-monitoring, beat/metrics-monitoring Actual (9): ...minus system/metrics-default and system/metrics-monitoring All components declare input\_spec.binary\_name: elastic-otel-collector (EDOT hosts everything). 1. The collector itself is healthy. collector.log is entirely info-level (3,344 lines, zero warn/error). Both system/metrics metricbeatreceivers are running and emitting "Non-zero metrics" every 30s: "otelcol.component.id":"metricbeatreceiver/\_agent-component/system/metrics-default" (1,408 lines) "otelcol.component.id":"metricbeatreceiver/\_agent-component/system/metrics-monitoring" (1,408 lines) "metricbeat":{"system":{"process":{"events":1,"success":1}}} So the metricbeat-via-OTEL receivers work at the collector level — the failure is purely on the agent-collector management plane (the agent can't register/manage those two components, which is why they're dropped from state.yaml/components-actual.yaml even though the collector keeps them alive). 1. The only non-telemetry lines in elastic-agent.log are these two warnings from internal/pkg/otel/manager/diagnostics.go:137: - component ID "system/metrics-default" contains '/', its EDOT diagnostics will be missing from the archive - component ID "system/metrics-monitoring" contains '/', its EDOT diagnostics will be missing from the archive The agent's OTel manager is flagging exactly these two component IDs as mishandled because they contain a / (the system/metrics input type embeds a slash). Notably, sibling IDs that also contain a slash (audit/file\_integrity-default, beat/metrics-monitoring) come up healthy — the differentiator is that the two failing ones are the system/metrics metricbeat-via-OTEL components. 1. This is a regression vs. the previous build. On the prior Agent 9.4.2 (same host, before reimage) the system/metrics-default / system/metrics-monitoring metricbeatreceivers worked (confirmed in older journald telemetry). The break appeared with the Ubuntu 24.04 reimage + Agent 9.4.4 upgrade. 2. Not the cause (ruled out): - Fleet connectivity: healthy (Connected, fleet\_state 2). - ES exporter auth: works (other components ack events). - metrics.log errors: only two "Exporting failed. Rejecting data" with "error":{"message":"context canceled"} at collector startup — a restart/reconfigure artifact coincident with a "stopping metricbeat receiver" line, not persistent. - No actual panics in panic.log (misnamed — it's all info-level telemetry). Questions for the community 1. Is the component ID … contains '/', its EDOT diagnostics will be missing from the archive warning in 9.4.4's OTel manager a known issue for system/metrics inputs under the metricbeat. default: otel runtime? It looks like the only two components failing to register are the system/metrics ones, and they're the exact IDs flagged. 2. Is the right fix to upgrade past 9.4.4 to a release that handles EDOT component IDs containing /, or is there a known workaround? 3. As a workaround, is it supported/recommended to route just the system/metrics inputs back off the OTEL runtime (set metricbeat.default away from otel, or via policy runtime setting) so they run as plain metricbeat subprocesses? Any caveats for a Fleet-managed agent? 4. The headline error is failed to connect to collector on the agent↔collector gRPC channel (localhost:6789, unix sockets under /opt/Elastic/Agent/data/tmp/\*.sock). Anything specific to Ubuntu 24.04 (systemd sandboxing, AppArmor, socket perms, PrivateTmp) known to break this channel that I should check? Any pointers would be hugely appreciated — happy to attach the full state.yaml / otel-merged.yaml / collector.log snippets if useful. ---
**Author:** ![DavidA](https://avatars.discourse-cdn.com/v4/letter/d/ba8739/32.png) [@DavidA](https://discuss.elastic.co/u/DavidA)\ **Post date:** [September 15, 2026, 7:09pm UTC](https://discuss.elastic.co/t/unhealthy-edot-otel-collector-in-agent-9-4-x/388928/2 "2026-09-15T19:09:34Z")
After much research and the assistance from another engineer. The issue came from where Elastic Agent and OTEL are looking for the metrics needed to populate CPU and memory, which would return the OTEL in the state of Unhealthy. Which all comes back to the /etc/hosts file reading the machines as 'localhost' instead of the actual hostname or FQDN. After updating the /etc/hosts file and removing the additional localhost from the 127.0.1.1 line, the agents and OTEL came back as 'Healthy' and reporting all proper metrics.