# Filebeat is not attaching lifecycle policy to indexes created

**URL:** <https://discuss.elastic.co/t/filebeat-is-not-attaching-lifecycle-policy-to-indexes-created/371525>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [December 5, 2024, 10:50am UTC](https://discuss.elastic.co/t/filebeat-is-not-attaching-lifecycle-policy-to-indexes-created/371525 "2024-12-05T10:50:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Saif\_Ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saif_ali/32/137626_2.png) [@Saif\_Ali](https://discuss.elastic.co/u/Saif_Ali)\
**Post date:** [December 5, 2024, 10:50am UTC](https://discuss.elastic.co/t/filebeat-is-not-attaching-lifecycle-policy-to-indexes-created/371525/1 "2024-12-05T10:50:33Z")

</div>

I use filebeat as a docker container in our eks cluster, which travels through private DNS to our another eks cluster which hosts ECK stack, i have setup everything the indexes are created the logs are pushing , data streams and views are setup. I want to setup 7 days index retention policy which is created already in kibana dashboard.

The issue is that i'm unable to attach the policy to each index created the logs policy only gets attached to filebeat index template only not to the other dev or prod indexes created.

Here is my

> filebeat.yml

```auto
filebeat.config:
  modules:
    path: /usr/share/filebeat/modules.d/*.yml
    reload.enabled: false

processors:
  - add_docker_metadata:
      host: "unix:///var/run/docker.sock"
      match_source: true
  - decode_json_fields:
      fields: ["message"]
      target: ""
      overwrite_keys: true
  - drop_fields:
      fields: 
        - "@timestamp"
        - "message.keyword"
        - "agent.ephemeral_id"
        - "agent.ephemeral_id.keyword"
        - "agent.id"
        - "agent.id.keyword"
        - "agent.name"
        - "agent.name.keyword"
        - "agent.type"
        - "agent.type.keyword"
        - "agent.version"
        - "agent.version.keyword"
        - agent
        - "ecs.version"
        - "host.name"
        - "input.type"
        - "log.file.path"
        - "log.offset"
        - "stream"
        - "_id"
        - "_index"
        - "_version"
        - "_score"
        - "fields"
        - "sort"

filebeat.inputs:
- type: container
  paths:
    - /var/log/containers/nbula-fe-*.log
  index: "${var.is_dev ? "dev" : "prod"}-nbula-fe-dev-%%{+yyyy.MM.dd}"
  lifecycle:
    name: "${var.is_dev ? "logs-policy-dev" : "logs-policy-prod"}"
    rollover_alias: ["rollover-${var.is_dev ? "dev" : "prod"}"]

- type: container 
  paths:
    - /var/log/containers/nbula-be-*.log
  index: "${var.is_dev ? "dev" : "prod"}-nbula-be-dev-%%{+yyyy.MM.dd}"
  lifecycle:
    name: "${var.is_dev ? "logs-policy-dev" : "logs-policy-prod"}"
    rollover_alias: ["rollover-${var.is_dev ? "dev" : "prod"}"]

- type: container
  paths:
    - /var/log/containers/nbula-ai-*.log  
  index: "${var.is_dev ? "dev" : "prod"}-nbula-ai-dev-%%{+yyyy.MM.dd}"
  lifecycle:
    name: "${var.is_dev ? "logs-policy-dev" : "logs-policy-prod"}"
    rollover_alias: ["rollover-${var.is_dev ? "dev" : "prod"}"]

- type: container
  paths:
    - /var/log/containers/nbula-mm-*.log
  index: "${var.is_dev ? "dev" : "prod"}-nbula-mm-dev-%%{+yyyy.MM.dd}"
  lifecycle:
    name: "${var.is_dev ? "logs-policy-dev" : "logs-policy-prod"}"
    rollover_alias: ["rollover-${var.is_dev ? "dev" : "prod"}"]

- type: container
  paths:
    - /var/log/containers/nxus-ls-app-*.log
  index: "${var.is_dev ? "dev" : "prod"}-nxus-ls-app-dev-%%{+yyyy.MM.dd}"
  lifecycle:
    name: "${var.is_dev ? "logs-policy-dev" : "logs-policy-prod"}"
    rollover_alias: ["rollover-${var.is_dev ? "dev" : "prod"}"]

output.elasticsearch:
  hosts: ["${var.elastic_domain}"]
  ssl.enabled: true
  username: "${var.elastic_username}"
  password: "${var.elastic_password}"

setup.ilm.enabled: true
setup.template.overwrite: true
setup.template.settings:
  index.lifecycle.name: "${var.is_dev ? "logs-policy-dev" : "logs-policy-prod"}"
  index.lifecycle.rollover_alias: "rollover-${var.is_dev ? "dev" : "prod"}"

kibana:
  enabled: true
  host: "${var.kibana_domain}"
  ssl:
    enabled: true
  username: "${var.elastic_username}"
  password: "${var.elastic_password}"

```

Please guide me what im doing wrong?

End goal is to have daily indexes of dev and prod services and dev logs should be retained each 7 days and prod for 30 days.

P.S the vars will be substituted.

Thanks

---

<div class="post-metadata">

**Author:** ![ashishtiwari1993](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashishtiwari1993/32/135241_2.png) [@ashishtiwari1993](https://discuss.elastic.co/u/ashishtiwari1993)\
**Post date:** [December 9, 2024, 5:46am UTC](https://discuss.elastic.co/t/filebeat-is-not-attaching-lifecycle-policy-to-indexes-created/371525/2 "2024-12-09T05:46:25Z")

</div>

Hi @Saif_Ali, Welcome to the Elastic community.

This more looks env vars issue. Can you [check](https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-get-lifecycle.html#ilm-get-lifecycle-request) whether the policy is created or not ? also can you just pass the static ILM in your configuration and check.