# Some documented fields (e.g. kubernetes.container.image) missing from events

**URL:** <https://discuss.elastic.co/t/some-documented-fields-e-g-kubernetes-container-image-missing-from-events/185217>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 11, 2019, 2:31pm UTC](https://discuss.elastic.co/t/some-documented-fields-e-g-kubernetes-container-image-missing-from-events/185217 "2019-06-11T14:31:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bagratte](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bagratte/32/37155_2.png) [@bagratte](https://discuss.elastic.co/u/bagratte)\
**Post date:** [June 11, 2019, 2:31pm UTC](https://discuss.elastic.co/t/some-documented-fields-e-g-kubernetes-container-image-missing-from-events/185217/1 "2019-06-11T14:31:18Z")

</div>

As per [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html#\_kubernetes](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html#_kubernetes), `kubernetes.container.image` is available in filebeat events.

Consider a deployment of filebeat on a Kubernetes cluster with the following manifest, which is the same as the manifest used in `curl` in [https://www.elastic.co/guide/en/beats/filebeat/current/running-on-kubernetes.html#\_kubernetes\_deploy\_manifests](https://www.elastic.co/guide/en/beats/filebeat/current/running-on-kubernetes.html#_kubernetes_deploy_manifests) with just `output.elasticsearch` replaced with `output.console`:

```auto
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: elastic-stack-test
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.config:
      inputs:
        # Mounted `filebeat-inputs` configmap:
        path: ${path.config}/inputs.d/*.yml
        # Reload inputs configs as they change:
        reload.enabled: false
      modules:
        path: ${path.config}/modules.d/*.yml
        # Reload module configs as they change:
        reload.enabled: false

    # To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
    #filebeat.autodiscover:
    # providers:
    # - type: kubernetes
    # hints.enabled: true

    processors:
      - add_cloud_metadata:

    cloud.id: ${ELASTIC_CLOUD_ID}
    cloud.auth: ${ELASTIC_CLOUD_AUTH}

    # output.elasticsearch:
    # hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
    # username: ${ELASTICSEARCH_USERNAME}
    # password: ${ELASTICSEARCH_PASSWORD}

    output.console:
      enabled: true
      codec.json:
        pretty: true
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-inputs
  namespace: elastic-stack-test
  labels:
    k8s-app: filebeat
data:
  kubernetes.yml: |-
    - type: docker
      containers.ids:
      - "*"
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: elastic-stack-test
  labels:
    k8s-app: filebeat
spec:
  template:
    metadata:
      labels:
        k8s-app: filebeat
    spec:
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      containers:
      - name: filebeat
        image: docker.elastic.co/beats/filebeat:7.1.1
        args: [
          "-c", "/etc/filebeat.yml",
          "-e",
        ]
        env:
        - name: ELASTICSEARCH_HOST
          value: elasticsearch
        - name: ELASTICSEARCH_PORT
          value: "9200"
        - name: ELASTICSEARCH_USERNAME
          value: elastic
        - name: ELASTICSEARCH_PASSWORD
          value: changeme
        - name: ELASTIC_CLOUD_ID
          value:
        - name: ELASTIC_CLOUD_AUTH
          value:
        securityContext:
          runAsUser: 0
          # If using Red Hat OpenShift uncomment this:
          #privileged: true
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 100Mi
        volumeMounts:
        - name: config
          mountPath: /etc/filebeat.yml
          readOnly: true
          subPath: filebeat.yml
        - name: inputs
          mountPath: /usr/share/filebeat/inputs.d
          readOnly: true
        - name: data
          mountPath: /usr/share/filebeat/data
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      volumes:
      - name: config
        configMap:
          defaultMode: 0600
          name: filebeat-config
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: inputs
        configMap:
          defaultMode: 0600
          name: filebeat-inputs
      # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
      - name: data
        hostPath:
          path: /var/lib/filebeat-data
          type: DirectoryOrCreate
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: filebeat
subjects:
- kind: ServiceAccount
  name: filebeat
  namespace: elastic-stack-test
roleRef:
  kind: ClusterRole
  name: filebeat
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: filebeat
  labels:
    k8s-app: filebeat
rules:
- apiGroups: [""] # "" indicates the core API group
  resources:
  - namespaces
  - pods
  verbs:
  - get
  - watch
  - list
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: filebeat
  namespace: elastic-stack-test
  labels:
    k8s-app: filebeat
---

```

Here is an event then emitted by filebeat:

```auto
{
  "@timestamp": "2019-06-11T14:21:37.797Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "7.1.1"
  },
  "log": {
    "offset": 2671026,
    "file": {
      "path": "/var/lib/docker/containers/acdfb39605861c0b170884efc0db53107f03897b5348d930e8ebfbfa41df7479/acdfb39605861c0b170884efc0db53107f03897b5348d930e8ebfbfa41df7479-json.log"
    }
  },
  "input": {
    "type": "docker"
  },
  "kubernetes": {
    "container": {
      "name": "filebeat"
    },
    "namespace": "elastic-stack-test",
    "labels": {
      "controller-revision-hash": "1750708910",
      "k8s-app": "filebeat",
      "pod-template-generation": "1"
    },
    "pod": {
      "name": "filebeat-kngrb",
      "uid": "37ea18f7-8c54-11e9-b3d3-0683a22eb04e"
    },
    "node": {
      "name": "ip-10-10-52-185.eu-central-1.compute.internal"
    }
  },
  "cloud": {
    "region": "eu-central-1",
    "availability_zone": "eu-central-1a",
    "instance": {
      "id": "i-0e5539ca7a497da8e"
    },
    "provider": "aws",
    "machine": {
      "type": "m5.large"
    }
  },
  "host": {
    "name": "filebeat-kngrb"
  },
  "stream": "stdout",
  "message": " },",
  "agent": {
    "hostname": "filebeat-kngrb",
    "id": "08806c22-3a7d-49e8-815b-516c9fc28463",
    "version": "7.1.1",
    "type": "filebeat",
    "ephemeral_id": "5531ccce-704e-4342-a69c-442a4b5caef1"
  },
  "ecs": {
    "version": "1.0.0"
  }
}

```

`kubernetes.container.image` (`kubernetes.container.id` too) is missing from the event.

I suspect this is a bug. Please confirm.

Thanks

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [June 13, 2019, 9:31am UTC](https://discuss.elastic.co/t/some-documented-fields-e-g-kubernetes-container-image-missing-from-events/185217/2 "2019-06-13T09:31:52Z")

</div>

Hi @bagratte,

These manifests don't use autodiscover by default, but a static `docker` input with `add_kubernetes_metadata`. See [https://www.elastic.co/guide/en/beats/filebeat/master/add-kubernetes-metadata.html](https://www.elastic.co/guide/en/beats/filebeat/master/add-kubernetes-metadata.html) for more details.

That said, both autodiscover and `add_kubernetes_metadata` provide the same metadata, and container image is not part of it. Is there anything you wanted to do with it? I'm curious about this use case.

Best regards

---

<div class="post-metadata">

**Author:** ![bagratte](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bagratte/32/37155_2.png) [@bagratte](https://discuss.elastic.co/u/bagratte)\
**Post date:** [June 13, 2019, 4:21pm UTC](https://discuss.elastic.co/t/some-documented-fields-e-g-kubernetes-container-image-missing-from-events/185217/3 "2019-06-13T16:21:13Z")

</div>

Hi @exekias,

If `kubernetes.container.image` is not part of the added metadata, first of all we shall remove any mentions of it from the docs.

But yes, `kubernetes.container.image` is a nice piece of useful metadata. I wanted to filter the events in Logstash based on this metadata, which in my case is the cleanest way because we have a couple of deployments of the same docker image, configured differently, that produce the same format of logs. Also, in general, `kubernetes.container.image` is a better selector for log filtering as opposed to `kubernetes.labels.some-label` or `kubernetes.container.name` or whatnot because the format of logs is determined by it.

Regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 11, 2019, 4:21pm UTC](https://discuss.elastic.co/t/some-documented-fields-e-g-kubernetes-container-image-missing-from-events/185217/4 "2019-07-11T16:21:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.