# Split index in runtime

**URL:** <https://discuss.elastic.co/t/split-index-in-runtime/291329>\
**Category:** Logstash\
**Created:** [December 9, 2021, 1:04pm UTC](https://discuss.elastic.co/t/split-index-in-runtime/291329 "2021-12-09T13:04:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![valentineg](https://avatars.discourse-cdn.com/v4/letter/v/6de8d8/32.png) [@valentineg](https://discuss.elastic.co/u/valentineg)\
**Post date:** [December 9, 2021, 1:04pm UTC](https://discuss.elastic.co/t/split-index-in-runtime/291329/1 "2021-12-09T13:04:25Z")

</div>

HI,

we have logstash and elastic on a k8s environment.  
in our topology logstash receives data from multiple applications while part of the data is application name \ id. is there a way to add a dynamic variable to the index name or alias and still maintain ILM?  
important to note we do not know all application names at logstash startup, and need to react based on arriving traffic

manipulating the "ilm\_rollover\_alias" or using "index" allows us to change index name but it brakes ILM

```auto
    output {
            ..
            elasticsearch {
                ilm_enabled => true
                ilm_rollover_alias => "events"
                ilm_pattern => "{now/d}-000001"
                ilm_policy => "policy_1"
           ..

```

we would like the index to be: "events--date-000001"

Appreciate your help!

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [December 9, 2021, 3:57pm UTC](https://discuss.elastic.co/t/split-index-in-runtime/291329/2 "2021-12-09T15:57:44Z")

</div>

When you define an ILM policy you define which template gets used.  
When you define a template you can define an index pattern that matches to any index based on that pattern.  
So instead of defining your ILM policy in logstash you can create a template in Elasticsearch that is already inherited by an ILM policy.  
Then when you write out to Elasticsearch write to an index that uses that template (index pattern matches) and then it should automatically be included in the ILM policy.

---

<div class="post-metadata">

**Author:** ![valentineg](https://avatars.discourse-cdn.com/v4/letter/v/6de8d8/32.png) [@valentineg](https://discuss.elastic.co/u/valentineg)\
**Post date:** [December 12, 2021, 8:01am UTC](https://discuss.elastic.co/t/split-index-in-runtime/291329/3 "2021-12-12T08:01:11Z")

</div>

@AquaX , thanks for your reply!

it actually makes a lot of sense.  
while we seem to have made progress, we're still struggling in the implementation.  
now the Logstash config looks like so:

```auto
elasticsearch {
  index => "event-%{param1}-000001"
  template_name => "event-tmpl"

```

so the indexes to look:

> event-\<app\_name\>-000001

in elastic the index template, "event-tmpl" has the following config:

```auto
{ 
 "index": {
  "lifecycle": {
  "name": "events_ilm",
  "rollover_alias": "event"
 },

```

now the problem is setting the rollover alias, seeing as the name is dynamic (it's located in the mapping) we can't seem to find the correct way to assign it.  
please note we have several active log sources and need them routed to different indexes, each should have ILM based on the "events\_ilm" policy.

thanks for the assist!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 9, 2022, 8:02am UTC](https://discuss.elastic.co/t/split-index-in-runtime/291329/4 "2022-01-09T08:02:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.