# Filebeat is not creating indices for each pod name

**URL:** <https://discuss.elastic.co/t/filebeat-is-not-creating-indices-for-each-pod-name/362715>\
**Category:** Beats\
**Tags:** elastic-stack-alerting, filebeat\
**Created:** [July 8, 2024, 1:21pm UTC](https://discuss.elastic.co/t/filebeat-is-not-creating-indices-for-each-pod-name/362715 "2024-07-08T13:21:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nandeesh\_b](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nandeesh_b/32/135887_2.png) [@nandeesh\_b](https://discuss.elastic.co/u/nandeesh_b)\
**Post date:** [July 8, 2024, 1:21pm UTC](https://discuss.elastic.co/t/filebeat-is-not-creating-indices-for-each-pod-name/362715/1 "2024-07-08T13:21:11Z")

</div>

Please help on this and stuck in this from last 5 days and not able to create index for each pod in Elasticsearch using filebeat.

Elastic search version: 7.10.1 AWS  
Filebeat version: 7.12.1

```auto
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: filebeat
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.inputs:
    - type: container
      paths:
        - /var/lib/kubelet/pods/*_communication-service_*
      tags: ["communication-service"]
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/lib/kubelet/pods/"
                resource_type: "pod"
            default_indexers.enabled: false
            default_matchers.enabled: false
            indexers:
              - pod_name:
    - type: container
      paths:
        - /var/lib/kubelet/pods/*_bgv-dl-verification_*
      tags: ["bgv-dl-verification"]
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/lib/kubelet/pods/"
                resource_type: "pod"
            default_indexers.enabled: false
            default_matchers.enabled: false
            indexers:
              - pod_name:
  
 output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:443}']
      username: ${ELASTICSEARCH_USERNAME}
      password: ${ELASTICSEARCH_PASSWORD}
      protocol: "https"
      indices:
      - index: "logs-%{[kubernetes.pod.name]}-%{+yyyy.MM.dd}"
        when.contains:
          tags: "communication-service"
      - index: "logs-%{[kubernetes.pod.name]}-%{+yyyy.MM.dd}"
        when.contains:
          tags: "bgv-dl-verification"
      - index: "logs-%{[kubernetes.pod.name]}-%{+yyyy.MM.dd}"
        when.contains:
          tags: "reports-generator"
      - index: "logs-%{[kubernetes.pod.name]}-%{+yyyy.MM.dd}"
        when.contains:

nd the file beat logs says index is created but dont see any index with name of the pod

    DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:206 Adding kubernetes pod: logging/fluentd-8cvhz {"libbeat.processor": "add_kubernetes_metadata"}
    2024-07-05T13:45:46.712Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:296 Created index logging/fluentd-8cvhz for pod logging/fluentd-8cvhz {"libbeat.processor": "add_kubernetes_metadata"}
    2024-07-05T13:45:46.712Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:206 Adding kubernetes pod: kube-system/ebs-csi-node-fkj9g {"libbeat.processor": "add_kubernetes_metadata"}
    2024-07-05T13:45:46.712Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:296 Created index kube-system/ebs-csi-node-fkj9g for pod kube-system/ebs-csi-node-fkj9g {"libbeat.processor": "add_kubernetes_metadata"}
    2024-07-05T13:45:46.712Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:206 Adding kubernetes pod: dev/insurance-service-dev-deployment-649c79c86d-nbrvw {"libbeat.processor": "add_kubernetes_metadata"}
    2024-07-05T13:45:46.718Z DEBUG [kubernetes] add_kubernetes_metadata/kubernetes.go:296 Created index dev/
```

---

<div class="post-metadata">

**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\
**Post date:** [July 8, 2024, 1:22pm UTC](https://discuss.elastic.co/t/filebeat-is-not-creating-indices-for-each-pod-name/362715/2 "2024-07-08T13:22:55Z")

</div>

Hi @nandeesh_b,

Welcome! To confirm, by Elasticsearch version 7.10.1 AWS, are you referring to the [Amazon Elasticsearch Service (now OpenSearch)](https://aws.amazon.com/about-aws/whats-new/2021/04/amazon-elasticsearch-service-announces-support-for-elasticsearch-version-7-10/) or a 7.10.1 Elasticsearch instance deployed to AWS?

It's an old version that is past EOL so I want to check.

---

<div class="post-metadata">

**Author:** ![nandeesh\_b](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nandeesh_b/32/135887_2.png) [@nandeesh\_b](https://discuss.elastic.co/u/nandeesh_b)\
**Post date:** [July 8, 2024, 2:53pm UTC](https://discuss.elastic.co/t/filebeat-is-not-creating-indices-for-each-pod-name/362715/3 "2024-07-08T14:53:55Z")

</div>

Hi @carly.richmond ,

Yes..Its AWS Opensearch --Elastic search 7.10 ..

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 8, 2024, 2:53pm UTC](https://discuss.elastic.co/t/filebeat-is-not-creating-indices-for-each-pod-name/362715/4 "2024-07-08T14:53:56Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance. See [What is OpenSearch and the OpenSearch Dashboard? | Elastic](https://www.elastic.co/elasticsearch/opensearch) for more details.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 8, 2024, 8:45pm UTC](https://discuss.elastic.co/t/filebeat-is-not-creating-indices-for-each-pod-name/362715/5 "2024-07-08T20:45:37Z")

</div>

You can't mix major versions. 7.12 with 7.10.

The best option is to at least use 7.17 which contains all the bug fixes and more importantly the security patches.

Better is to go to 8.14.2. It's also available from the AWS marketplace.
