# Labels and Annotation are not applied at the pod level

**URL:** https://discuss.elastic.co/t/labels-and-annotation-are-not-applied-at-the-pod-level/262522
**Category:** Elastic Cloud on Kubernetes (ECK)
**Created:** [January 28, 2021, 3:00pm UTC](https://discuss.elastic.co/t/labels-and-annotation-are-not-applied-at-the-pod-level/262522 "2021-01-28T15:00:00Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![dg\_hivebrite](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dg_hivebrite/32/46359_2.png) [@dg\_hivebrite](https://discuss.elastic.co/u/dg_hivebrite)
#### Post date: [January 28, 2021, 3:00pm UTC](https://discuss.elastic.co/t/labels-and-annotation-are-not-applied-at-the-pod-level/262522/1 "2021-01-28T15:00:00Z")

</div>

Hi,

I'm trying to do a poc with datadog to monitor elasticsearch.  
One of the solution is to setting annotations and labels on pods.  
So I tried to do that, using the `podTemplate` field in the CRD but it doesn't update the stateful set.  
I have to following manifest:

```auto
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  annotations:
    common.k8s.elastic.co/controller-version: 1.3.0
    elasticsearch.k8s.elastic.co/cluster-uuid: R-x_d2qVQO6VkfjLvauX8w
    meta.helm.sh/release-name: eck-elastic-alumni-connect
    meta.helm.sh/release-namespace: default
  creationTimestamp: "2020-11-10T06:23:04Z"
  generation: 8
  labels:
    app.kubernetes.io/managed-by: Helm
  name: alumni-connect
  namespace: default
  resourceVersion: "222736386"
  selfLink: /apis/elasticsearch.k8s.elastic.co/v1/namespaces/default/elasticsearches/alumni-connect
  uid: ec916938-0802-489a-bbb1-9cee580d9c59
spec:
  auth:
    fileRealm:
    - secretName: elk-users-secret
    roles:
    - secretName: elk-roles-secret
  http:
    service:
      metadata:
        creationTimestamp: null
      spec: {}
    tls:
      certificate: {}
      selfSignedCertificate:
        disabled: true
  image: gcr.io/hivebrite/elasticsearch7:481d0cda7
  nodeSets:
  - config:
      node.data: true
      node.master: true
    count: 3
    name: all
    podTemplate:
      spec:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
              - matchExpressions:
                - key: workloadType
                  operator: In
                  values:
                  - elasticsearch3
          podAntiAffinity:
            preferredDuringSchedulingIgnoredDuringExecution:
            - podAffinityTerm:
                labelSelector:
                  matchExpressions:
                  - key: workloadName
                    operator: In
                    values:
                    - alumni-connect
                  - key: workloadScope
                    operator: In
                    values:
                    - alumni_connect
                  - key: workloadStack
                    operator: In
                    values:
                    - alumni_connect
                topologyKey: kubernetes.io/hostname
              weight: 100
        containers:
        - env:
          - name: ES_JAVA_OPTS
            value: -Xms4028m -Xmx4028m
          name: elasticsearch
          resources:
            limits:
              memory: 5000Mi
            requests:
              cpu: 1000m
              memory: 5000Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
              - SETPCAP
              - MKNOD
              - AUDIT_WRITE
              - CHOWN
              - NET_RAW
              - DAC_OVERRIDE
              - FOWNER
              - FSETID
              - KILL
              - SETGID
              - SETUID
              - NET_BIND_SERVICE
              - SYS_CHROOT
              - SETFCAP
            runAsUser: 1000
        initContainers:
        - command:
          - sh
          - -c
          - sysctl -w vm.max_map_count=262144
          name: sysctl
          securityContext:
            privileged: true
        metadata:
          annotations:
            ad.datadoghq.com/elastic.check_names: '["elastic"]'
            ad.datadoghq.com/elastic.init_configs: '[{}]'
            ad.datadoghq.com/elastic.instances: |
              [
                {
                  "url": "%%host%%",
                  "username": "elastic"
                  "password":"ENC[kubernetes_default_alumni-connect-es-elastic-user_elastic]",
                  "auth_type": "basic",
                  "cluster_stats": true
                  "tags": {
                    "env": "wip",
                    "service": "alumni-connect",
                    "version": "7.9.3"
                  }
                }
              ]
          labels:
            clusterName: wip
            region: europe-west1
            workloadName: alumni-connect
            workloadScope: alumni_connect
            workloadStack: alumni_connect
        tolerations:
        - key: workloadType
          operator: Equal
          value: elasticsearch3
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 500Gi
        storageClassName: ssd
  secureSettings:
  - secretName: alumni-connect-eck-gcp
  transport:
    service:
      metadata:
        creationTimestamp: null
      spec: {}
  updateStrategy:
    changeBudget:
      maxSurge: 1
      maxUnavailable: 1
  version: 7.9.3
status:
  availableNodes: 3
  health: green
  phase: Ready
  version: 7.9.3

```

and my statefulset manifest is:

```auto
apiVersion: apps/v1
kind: StatefulSet
metadata:
  creationTimestamp: "2020-11-10T06:23:09Z"
  generation: 7
  labels:
    common.k8s.elastic.co/template-hash: "90437220"
    common.k8s.elastic.co/type: elasticsearch
    elasticsearch.k8s.elastic.co/cluster-name: alumni-connect
    elasticsearch.k8s.elastic.co/statefulset-name: alumni-connect-es-all
  name: alumni-connect-es-all
  namespace: default
  ownerReferences:
  - apiVersion: elasticsearch.k8s.elastic.co/v1
    blockOwnerDeletion: true
    controller: true
    kind: Elasticsearch
    name: alumni-connect
    uid: ec916938-0802-489a-bbb1-9cee580d9c59
  resourceVersion: "222725688"
  selfLink: /apis/apps/v1/namespaces/default/statefulsets/alumni-connect-es-all
  uid: e73aeec5-572a-40d8-9a8f-68d6bd29970a
spec:
  podManagementPolicy: Parallel
  replicas: 3
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      common.k8s.elastic.co/type: elasticsearch
      elasticsearch.k8s.elastic.co/cluster-name: alumni-connect
      elasticsearch.k8s.elastic.co/statefulset-name: alumni-connect-es-all
  serviceName: alumni-connect-es-all
  template:
    metadata:
      annotations:
        co.elastic.logs/module: elasticsearch
      creationTimestamp: null
      labels:
        common.k8s.elastic.co/type: elasticsearch
        elasticsearch.k8s.elastic.co/cluster-name: alumni-connect
        elasticsearch.k8s.elastic.co/config-hash: "2296861737"
        elasticsearch.k8s.elastic.co/http-scheme: http
        elasticsearch.k8s.elastic.co/node-data: "true"
        elasticsearch.k8s.elastic.co/node-ingest: "true"
        elasticsearch.k8s.elastic.co/node-master: "true"
        elasticsearch.k8s.elastic.co/node-ml: "true"
        elasticsearch.k8s.elastic.co/node-transform: "true"
        elasticsearch.k8s.elastic.co/secure-settings-hash: 5477e82f85823f08a2712e4dad1f964bf1d28b419e8b00466cd2d3b9
        elasticsearch.k8s.elastic.co/statefulset-name: alumni-connect-es-all
        elasticsearch.k8s.elastic.co/version: 7.9.3
    ....

```

My cluster seems ok:

```auto
kubectl get elasticsearches.elasticsearch.k8s.elastic.co alumni-connect
NAME HEALTH NODES VERSION PHASE AGE
alumni-connect green 3 7.9.3 Ready 79d

```

The version of the operator is: 1.3.0

Someone have an idea why this conf is not working ? or How to apply labels and annotations at the pod labels

Best regards,  
Damien

---

<div class="post-metadata">

### Author: ![charith-elastic](https://avatars.discourse-cdn.com/v4/letter/c/3ec8ea/32.png) [@charith-elastic](https://discuss.elastic.co/u/charith-elastic)
#### Post date: [January 28, 2021, 4:08pm UTC](https://discuss.elastic.co/t/labels-and-annotation-are-not-applied-at-the-pod-level/262522/2 "2021-01-28T16:08:05Z")

</div>

You have added the annotations and labels under `podTemplate.spec` whereas they should be added under `podTemplate.metadata`. For example:

```auto
---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: hulk
spec:
  version: 7.10.0
  nodeSets:
    - name: default
      count: 3
      podTemplate:
        metadata:
          annotations:
            my-annotation: "annotation-value"
          labels:
            my-label: "label-value"

```

---

<div class="post-metadata">

### Author: ![dg\_hivebrite](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dg_hivebrite/32/46359_2.png) [@dg\_hivebrite](https://discuss.elastic.co/u/dg_hivebrite)
#### Post date: [January 28, 2021, 5:47pm UTC](https://discuss.elastic.co/t/labels-and-annotation-are-not-applied-at-the-pod-level/262522/3 "2021-01-28T17:47:15Z")

</div>

Good catch, it works 🎉  
Thank you for the help

Regards,  
Damien

---

<div class="post-metadata">

### Author: ![kirkiris](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kirkiris/32/91023_2.png) [@kirkiris](https://discuss.elastic.co/u/kirkiris)
#### Post date: [June 30, 2021, 3:30pm UTC](https://discuss.elastic.co/t/labels-and-annotation-are-not-applied-at-the-pod-level/262522/4 "2021-06-30T15:30:12Z")

</div>

Not relevant with the ECK operator but I came across this thread and I was wondering how you were able to configure the DD agent with the elastic user's password.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 4, 2022, 8:22am UTC](https://discuss.elastic.co/t/labels-and-annotation-are-not-applied-at-the-pod-level/262522/5 "2022-11-04T08:22:46Z")

</div>


