ECE Monitoring - cgroup memory fields mapped as keyword instead of numeric

Elastic Observability Metrics
1

Hello,

I'm investigating CPU throttling and resource entitlement for Elasticsearch nodes running on Elastic Cloud Enterprise (ECE).

While building dashboards from the `.monitoring-es-*` indices, I noticed the following fields are mapped as `keyword`:

  • `elasticsearch.node.stats.os.cgroup.memory.limit.bytes`
  • `elasticsearch.node.stats.os.cgroup.memory.usage.bytes`

Example field caps output:

"elasticsearch.node.stats.os.cgroup.memory.limit.bytes": {
  "keyword": {
    "type": "keyword"
  }
},
"elasticsearch.node.stats.os.cgroup.memory.usage.bytes": {
  "keyword": {
    "type": "keyword"
  }
}

The values themselves appear to be numeric byte counts (e.g. 34359738368 for 32 GB), but because they are mapped as keywords they cannot be used directly in Lens metrics, formulas, or byte formatting without runtime fields.

Is this expected behaviour for ECE/Metricbeat monitoring data, or is this a mapping issue?

Environment:

  • Elastic Cloud Enterprise (ECE) 4.0.3
  • Elasticsearch 9.4.2
  • Data source: .monitoring-es-mb-*

Best regards,

Willem

2

Hello!

Yes, this appears to be expected behavior rather than a mapping issue.

The fields:

elasticsearch.node.stats.os.cgroup.memory.limit.bytes

elasticsearch.node.stats.os.cgroup.memory.usage.bytes

are documented by Metricbeat as keyword fields, even though they contain numeric byte values:

Because of that, Kibana Lens cannot use them directly for numeric aggregations, formulas, or byte formatting without converting them first.

For dashboarding purposes, you can use runtime fields to cast the values to long, which is the approach recommended for adding calculated fields without reindexing existing data:

Based on the documented field definitions, seeing these fields mapped as keyword in .monitoring-es-mb-* is expected behavior and does not appear to be specific to ECE 4.0.3 or Elasticsearch 9.4.2.

One additional point worth checking is whether newer monitoring integrations (Elastic Agent / Stack Monitoring) expose equivalent metrics with numeric mappings, as the legacy Metricbeat monitoring indices have historically contained several fields stored as strings.

3

Thank you for the clarification.

I am still struggling to understand why mapping these cgroup memory fields as "keyword" is considered acceptable.

My concern is that fields such as:

- "system.process.cgroup.memory.mem.limit.bytes"

- "system.process.cgroup.memory.mem.usage.bytes"

- "system.process.cgroup.memory.memsw.usage.bytes"

- "system.process.cgroup.memory.stats.*"

represent quantitative metrics rather than identifiers.

For most observability use cases, I would expect users to:

- Create Lens visualizations showing memory usage over time

- Build threshold-based alerts

- Calculate averages, percentiles, maximums, and trends

- Use ES|QL or aggregations on the numeric values

- Derive utilization percentages (e.g. usage / limit)

When these fields are mapped as "keyword", Kibana treats them as strings rather than metrics, which prevents many of these use cases without additional runtime fields or ingest-time conversion.

I understand the rationale for mapping identifier-like values as "keyword" when only exact matching is required, as described in the Elasticsearch documentation. However, cgroup memory values appear fundamentally different from identifiers because they represent measurements that are naturally consumed as numeric telemetry.

Could you help me understand:

1. Is the expectation that users should create runtime fields for these metrics?

2. Is there a technical reason these fields cannot be safely mapped as "long"?

3. Is there any roadmap to expose these values as numeric fields in future ECE monitoring integrations?

From a monitoring and capacity-planning perspective, having memory-related metrics stored as numeric values would seem more aligned with how they are typically analyzed in Elasticsearch and Kibana.

I'd appreciate some additional background on the design decision.

Kind regards,

Willem

4

I think your questions are completely valid.

From an observability perspective, metrics such as cgroup memory usage and limits are naturally expected to be numeric, since common use cases include aggregations, thresholds, Lens visualizations, ES|QL queries, and utilization calculations.

The Metricbeat documentation confirms that these fields are currently defined as keyword:

However, I couldn’t find any official documentation explaining why these fields were intentionally mapped as keyword instead of numeric types. That design rationale does not appear to be documented.

Because of that, I don’t think anyone outside the Elastic engineering team can definitively answer whether this was:

  • an intentional design decision,
  • a legacy implementation,
  • a compatibility constraint,
  • or simply something that has never been revisited.

Since your question is really about the design rationale rather than how to work around it, I would recommend opening a GitHub issue in the Beats (or Integrations) repository.

That would allow the engineering team to explain the original reasoning and, if appropriate, evaluate whether exposing these fields as numeric types would make sense for future monitoring integrations.

I think this is a reasonable enhancement request, especially considering the typical observability use cases you’ve described.

5

Thanks for the feedback @Rafa_Silva

I made Monitoring indices map cgroup memory metrics as keyword instead of numeric · Issue #51637 · elastic/beats · GitHub