okay, thank you
hello
@jsoriano
I found container.id after upgrading filebeat to 7.9.2, but container.id still does not exist in metricbeat version 7.9.2. Will you add container.id next time?
Hey @wajika,
container.id field should be also present in events created by Metricbeat 7.9.2.
One thing that might be happening is that your configuration is only matching the pod events, that don't contain information about specific containers (a pod can contain multiple containers, sharing the same network namespace). How is the autodiscover configuration you are using in Metricbeat?
Why do other namespace data appear?
apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-daemonset-config namespace: kube-system labels: k8s-app: metricbeat data: metricbeat.yml: |- metricbeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: true metricbeat.autodiscover: providers: - type: kubernetes templates: - condition: or: - equals: kubernetes.namespace: back - equals: kubernetes.namespace: front processors: - add_docker_metadata: host: "unix:///var/run/docker.sock" - add_kubernetes_metadata: in_cluster: true host: ${NODE_NAME} output.elasticsearch: hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}'] --- apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-fields-config namespace: kube-system labels: k8s-app: metricbeat data: fields.yml: |- - name: service type: group description: > kubernetes service metrics release: experimental fields: - name: name type: keyword description: Service name. - name: cluster_ip type: keyword description: Internal IP for the service. - name: external_name type: keyword description: Service external DNS name - name: external_ip type: keyword description: Service external IP - name: load_balancer_ip type: keyword description: Load Balancer service IP - name: type type: keyword description: Service type - name: ingress_ip type: keyword description: Ingress IP - name: ingress_hostname type: keyword description: Ingress Hostname - name: created type: date description: Service creation date --- apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-daemonset-config namespace: kube-system labels: k8s-app: metricbeat data: metricbeat.yml: |- metricbeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: true metricbeat.autodiscover: providers: - type: kubernetes templates: - condition: or: - equals: kubernetes.namespace: back - equals: kubernetes.namespace: front processors: - add_docker_metadata: host: "unix:///var/run/docker.sock" - add_kubernetes_metadata: in_cluster: true host: ${NODE_NAME} processors: - add_docker_metadata: host: "unix:///var/run/docker.sock" - add_kubernetes_metadata: output.elasticsearch: hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}'] --- apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-daemonset-modules namespace: kube-system labels: k8s-app: metricbeat data: system.yml: |- - module: system period: 10s metricsets: - cpu - load - memory - network - process - process_summary #- core #- diskio #- socket processes: ['.*'] process.include_top_n: by_cpu: 5 # include top 5 processes by CPU by_memory: 5 # include top 5 processes by memory - module: system period: 1m metricsets: - filesystem - fsstat processors: - drop_event.when.regexp: system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)' kubernetes.yml: |- - module: kubernetes metricsets: - node - system - pod - container - volume period: 10s enabled: true host: ${NODE_NAME} hosts: ["https://${NODE_NAME}:10250"] bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token ssl.verification_mode: "none" - module: kubernetes metricsets: - proxy period: 10s host: ${NODE_NAME} hosts: ["localhost:10249"] --- apiVersion: apps/v1 kind: DaemonSet metadata: name: metricbeat namespace: kube-system labels: k8s-app: metricbeat spec: selector: matchLabels: k8s-app: metricbeat template: metadata: labels: k8s-app: metricbeat spec: serviceAccountName: metricbeat terminationGracePeriodSeconds: 30 hostNetwork: true dnsPolicy: ClusterFirstWithHostNet containers: - name: metricbeat image: metricbeat:master-SNAPSHOT args: [ "-c", "/etc/metricbeat.yml", "-e", "-system.hostfs=/hostfs", ] env: - name: ELASTICSEARCH_HOST value: 192.168.10.145 - name: ELASTICSEARCH_PORT value: "9200" - name: ELASTICSEARCH_USERNAME value: - name: ELASTICSEARCH_PASSWORD value: - name: ELASTIC_CLOUD_ID value: - name: ELASTIC_CLOUD_AUTH value: - name: NODE_NAME valueFrom: fieldRef: fieldPath: status.hostIP securityContext: runAsUser: 0 resources: limits: memory: 200Mi requests: cpu: 100m memory: 100Mi volumeMounts: - name: config mountPath: /etc/metricbeat.yml readOnly: true subPath: metricbeat.yml - name: data mountPath: /usr/share/metricbeat/data - name: modules mountPath: /usr/share/metricbeat/modules.d readOnly: true - name: dockersock mountPath: /var/run/docker.sock - name: proc mountPath: /hostfs/proc readOnly: true - name: cgroup mountPath: /hostfs/sys/fs/cgroup readOnly: true volumes: - name: proc hostPath: path: /proc - name: cgroup hostPath: path: /sys/fs/cgroup - name: dockersock hostPath: path: /var/run/docker.sock - name: config configMap: defaultMode: 0600 name: metricbeat-daemonset-config - name: modules configMap: defaultMode: 0600 name: metricbeat-daemonset-modules - name: data hostPath: path: /var/lib/metricbeat-data type: DirectoryOrCreate --- apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-deployment-config namespace: kube-system labels: k8s-app: metricbeat data: metricbeat.yml: |- metricbeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: true metricbeat.autodiscover: providers: - type: kubernetes templates: - condition: or: - equals: kubernetes.namespace: back - equals: kubernetes.namespace: front processors: - add_docker_metadata: host: "unix:///var/run/docker.sock" - add_kubernetes_metadata: output.elasticsearch: hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}'] --- apiVersion: v1 kind: ConfigMap metadata: name: metricbeat-deployment-modules namespace: kube-system labels: k8s-app: metricbeat data: kubernetes.yml: |- - module: kubernetes metricsets: - state_node - state_deployment - state_replicaset - state_statefulset - state_pod - state_container - state_cronjob - state_resourcequota - state_service - state_persistentvolume - state_persistentvolumeclaim - state_storageclass # Uncomment this to get k8s events: #- event period: 10s #add_metadata: true host: ${NODE_NAME} hosts: ["kube-state-metrics:8080"] --- apiVersion: apps/v1 kind: Deployment metadata: name: metricbeat namespace: kube-system labels: k8s-app: metricbeat spec: selector: matchLabels: k8s-app: metricbeat template: metadata: labels: k8s-app: metricbeat spec: serviceAccountName: metricbeat hostNetwork: true dnsPolicy: ClusterFirstWithHostNet containers: - name: metricbeat image: metricbeat:master-SNAPSHOT args: [ "-c", "/etc/metricbeat.yml", "-e", ] env: - name: ELASTICSEARCH_HOST value: 192.168.10.145 - name: ELASTICSEARCH_PORT value: "9200" - name: ELASTICSEARCH_USERNAME value: - name: ELASTICSEARCH_PASSWORD value: - name: ELASTIC_CLOUD_ID value: - name: ELASTIC_CLOUD_AUTH value: - name: NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName securityContext: runAsUser: 0 resources: limits: memory: 200Mi requests: cpu: 100m memory: 100Mi volumeMounts: - name: config mountPath: /etc/metricbeat.yml readOnly: true subPath: metricbeat.yml - name: fields-config mountPath: /usr/share/metricbeat/fields.yml readOnly: true subPath: fields.yml - name: modules mountPath: /usr/share/metricbeat/modules.d readOnly: true volumes: - name: config configMap: defaultMode: 0600 name: metricbeat-deployment-config - name: fields-config configMap: defaultMode: 0600 name: metricbeat-fields-config - name: modules configMap: defaultMode: 0600 name: metricbeat-deployment-modules
Due to the mapping problem of some fields, I temporarily use the master tag.
about container.id , I can only see it on kibana metric UI, but not on discover.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.