Kubernetes Module of Metricbeat does not get pct metrics from Kubelet on Windows

[rituzzzpilot]

Hi, everyone

I have been testing metricbeat 7.17 Kubernetes module with Kubelet 1.24.3 and windows worker nodes (Windows Server 2022). I use Metricbeat in order to get metrics of containers, pods and nodes. I got node and pods metrics but kubernetes.pod.cpu.usage.node.pct and other pct is absent.
I use standard configuration ans all available metricsets, also I've tried run metricmeat locally and as a pod - same issue

[ChrsMark (Chris Mark)]

Hi!

Do you have access to the Kubelet's API to see what metrics it serves?
(I use sth like curl -H "Authorization: Bearer $token" https://${HOSTNAME}:10250/stats/summary --insecure)

Also make sure your configuration is aligned with what we suggest at Run Metricbeat on Kubernetes | Metricbeat Reference [7.17] | Elastic

[rituzzzpilot]

It seems to be ok

“containers”: [
    {
     “name”: “metricbeat-windows”,
     “startTime”: “2022-09-28T16:27:30Z”,
     “cpu”: {
      “time”: “2022-09-28T17:12:05Z”,
      “usageNanoCores”: 13926978,
      “usageCoreNanoSeconds”: 21734375000
     },
     “memory”: {
      “time”: “2022-09-28T17:12:05Z”,
      “workingSetBytes”: 60731392
     },
     “rootfs”: {
      “time”: “2022-09-28T17:11:57Z”,
      “availableBytes”: 15484485632,
      “capacityBytes”: 42275434496,
      “usedBytes”: 37748736,
      “inodesUsed”: 0
     },
     “logs”: {
      “time”: “2022-09-28T17:12:05Z”,
      “availableBytes”: 15484485632,
      “capacityBytes”: 42275434496,
      “usedBytes”: 1690,
      “inodesUsed”: 0

config if also fine:

metricbeat.modules:
- module: system
  period: 10s
  metricsets:
    - cpu
    - memory
    - network
    - process
    - process_summary
    - diskio
    - socket_summary

- module: system
  period: 1m
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

- module: kubernetes
  metricsets:
    - node
    - system
    - pod
    - container
    - volume
  period: 10s
  host: ${NODE_NAME}
  hosts: ["https://${NODE_NAME}:10250"]
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  add_metadata: true
  in_cluster: true
  labels.dedot: true
  annotations.dedot: true
  ssl.verification_mode: "none"

name: ${NODE_NAME}

processors:
- add_fields:
    target: ''
    fields:
      cluster_name: ${CLUSTER_NAME}


output.elasticsearch:
  hosts: ["elk_host:9200"]
  template.enabled: false
  index: "metricbeat_k8s-%{+yyyy.MM.dd}"
setup.template.enabled: false
logging.metrics.enabled: false
setup.ilm.enabled: false

And I have all needed metrics on linux nodes, the problem only on windows workers

[rituzzzpilot]

The issue solved by container rebuild