# How to attach Index Lifecycle Policy to indices created by Logstash?

**URL:** https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657
**Category:** Logstash
**Tags:** ilm-index-lifecycle-management
**Created:** [April 5, 2022, 6:14pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657 "2022-04-05T18:14:31Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![pritster5](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pritster5/32/103311_2.png) [@pritster5](https://discuss.elastic.co/u/pritster5)
#### Post date: [April 5, 2022, 6:14pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657/1 "2022-04-05T18:14:32Z")

</div>

I have a Logstash Config file that looks like this:

```auto
input {
    s3 {
        bucket => "testbucket"
        region => "us-east-1"
        codec => "json"
        additional_settings => {
            force_path_style => true
            follow_redirects => false
        }
    }
}

output {
    elasticsearch {
        hosts => "http://localhost:9200"
        index => "test.xxx.blahblahblah.output-%{+YYYY.MM.dd}"
        user => xxx
        password => yyy
    }
}

```

I want all indices created by Logstash (a new one is created daily) to obey an Index Lifecycle Policy I have created which looks like this:

```auto
PUT _ilm/policy/index-loop
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "14d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

How would I do this?

I currently do:

```auto
PUT test*/_settings 
{
  "index": {
    "lifecycle": {
      "name": "index-loop"
    }
  }
}

```

which matches against all the indices created by logstash, but I don't know if this will apply to indices created AFTER this put request was run.

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [April 5, 2022, 7:24pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657/2 "2022-04-05T19:24:31Z")

</div>

Do you have a template for your index?

You will need to create a template for your index and add the lifecycle policy in it.

Something like this:

```auto
  "index": {
    "lifecycle": {
      "name": "policy-name"
    }

```

---

<div class="post-metadata">

### Author: ![pritster5](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pritster5/32/103311_2.png) [@pritster5](https://discuss.elastic.co/u/pritster5)
#### Post date: [April 5, 2022, 8:58pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657/3 "2022-04-05T20:58:25Z")

</div>

I do not. How do I make an index template for an index created by logstash?

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [April 5, 2022, 9:20pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657/4 "2022-04-05T21:20:47Z")

</div>

You just need to create an index template in Elasticsearch and set the index\_pattern of the template to match the names that you are using in logstash.

For example, using the index\_pattern of `test.*` will match any index created which name starts with `test.`

The documentation about index templates is [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html)

---

<div class="post-metadata">

### Author: ![pritster5](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pritster5/32/103311_2.png) [@pritster5](https://discuss.elastic.co/u/pritster5)
#### Post date: [April 7, 2022, 4:25pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657/5 "2022-04-07T16:25:48Z")

</div>

If I'm making the index template through Kibana's GUI, would I put the

```auto
"index": {
    "lifecycle": {
      "name": "index-loop"
    }

```

inside here?:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/7/6755735d4af281839d07f05d2a89ed48bc1e8d8e.png)

And is the index policy itself stored internally after I execute?:

```auto
PUT _ilm/policy/index-loop
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "14d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![pritster5](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pritster5/32/103311_2.png) [@pritster5](https://discuss.elastic.co/u/pritster5)
#### Post date: [April 21, 2022, 9:40pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657/7 "2022-04-21T21:40:05Z")

</div>

SOLUTION (for ELK 7.5 at least):

1. Create an Index Template with an Index Pattern that matches the Index name defined in your Logstash pipeline config.

2. Make an Index Lifecycle Management Policy via Kibana GUI: Settings -\> Elasticsearch -\> Index Lifecycle Policies: Create Policy. Configure however you want.

3. Go back to the main Index Policy Menu, and click "Actions" next to the Index Policy you just made.

4. Click "Add policy to index template" and then select the index template we created in step 1.

5. Done. Any new indices created by Logstash that match the index pattern in your index template will automatically have the lifecycle policy we made applied to them.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 19, 2022, 9:40pm UTC](https://discuss.elastic.co/t/how-to-attach-index-lifecycle-policy-to-indices-created-by-logstash/301657/8 "2022-05-19T21:40:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
