Create Index Lifecycle Policies to existing index

Elastic Stack Kibana
1

Hi,
im trying to add 'Index Lifecycle Policies' to my existing index.
my steps:
Index management --> choose my index --> add lifecycle policy --> slect logs
but i got
'Policy logs is configured for rollover, but index my_index does not have an alias, which is required for rollover.'
how did i add an alias to my index?and what should be the alias?

2

Does Configure a lifecycle policy | Elasticsearch Guide [7.15] | Elastic help?

3

i already tried to do it, i created a template but still don't understand what should be the alias...

4

It might be useful if you shared the settings you are using.

5

image

so,
i tried to create my own 'index template' but still:

  1. don't know what should be the alias
    2.did my index template need to be rollover? how to disable this option if not?
6

Please share your full config.

7

I had same problem when I first started setting this up. didn't understand it.

It is three step proces. create ILM, create template, create index


## create ILM
PUT _ilm/policy/my_monitoring
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "30d",
            "max_size": "50gb"
          },
          "set_priority": {
            "priority": 100
          }
        }
      }
    }
  }
}

## Create template, you can change this latter for shard etc..
PUT _index_template/my_monitoring
{
  "index_patterns": ["my_monitoring-*"],
  "template": {
    "settings": {
      "number_of_shards": 3,
      "number_of_replicas": 0,
      "index.lifecycle.name": "my_monitoring",
      "index.lifecycle.rollover_alias": "my_monitoring"
    }
  }
}

## Create alias
## after this write your data only to my_monitoring and it will create proper date-000001 index
## This actually creates blank index my_monitoring-2021-07-14-000001

# PUT <my-index-{now/d}-000001>
PUT %3Cmy_monitoring-%7Bnow%2Fd%7D-000001%3E
{
  "aliases": {
    "my_monitoring": {
      "is_write_index": true
    }
  }
}

What will happen here is that, it will create new index called my_monitoring-date-000001
and in your logstash/python whereever you now use my_monitoring to write to.

and ILM will rotate index when time comes and will create new index called my_monitoring-date-000002 but your logstash/python will keep writing to my_monitoring

now basically my_monitoring -> my_monitoring-date-000002 ( which was 01 before) ILM will do this for you.

8

Thanks a lot!!!
im having some issues:
i followd all your steps and when trying to set the alias (step #3), i got an error 'an index or data stream exists with the same name as the alias'
so i changed the alias name and its worked.
but now i got the following error:

image
image897×125 12.8 KB

9

because you already have that index exist.

is this brand new setup or setting up in existing index?

10

existing one

11

ok then give me more info
what is your existing index name?

what are the command you executed? full list

for testing you can run all three code with some dummy indexname. once you satisfy and understand this you can remove ILM, and index and alias.

12

ok so,
index name: jenkins_jobs_logs
the commands are the same as you mentioned above just replace 'my_monitoring' with 'jenkins_jobs_logs':

## create ILM
PUT _ilm/policy/jenkins_jobs_logs
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "30d",
            "max_size": "50gb"
          },
          "set_priority": {
            "priority": 100
          }
        }
      }
    }
  }
}

## Create template, you can change this latter for shard etc..
PUT _index_template/my_monitoring
{
  "index_patterns": ["jenkins_jobs_logs-*"],
  "template": {
    "settings": {
      "number_of_shards": 3,
      "number_of_replicas": 0,
      "index.lifecycle.name": "jenkins_jobs_logs",
      "index.lifecycle.rollover_alias": "jenkins_jobs_logs"
    }
  }
}

## Create alias
## after this write your data only to my_monitoring and it will create proper date-000001 index
## This actually creates blank index my_monitoring-2021-07-14-000001

# PUT <my-index-{now/d}-000001>
PUT %3Cjenkins_jobs_logs-%7Bnow%2Fd%7D-000001%3E
{
  "aliases": {
    "jenkins_jobs_logs": {
      "is_write_index": true
    }
  }
}
13

and what is index name you have in your index management window?

14

the same name - 'jenkins_jobs_logs'

15

that is your problem. your index should be jenkins_jobs_logs--000001
because jenkins_jobs_logs is now alias points to
jenkins_jobs_logs -> jenkins_jobs_logs--000001

and when ILM runs it will create new index called "jenkins_jobs_logs--000002
and now your alias will pint to new index and old index will become read only
jenkins_jobs_logs -> jenkins_jobs_logs--000002

and you only write to one "jenkins_job_logs"

16

ok so all i need is to rename my index to jenkins_jobs_logs-000001 ?

17

yes

also check
GET /_cat/aliases and you should see jenkins_jobs_logs

18

thanks a lot!!!

19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.