ILM policy for custom index

Elastic Stack Elasticsearch
1

Hello,

I am unable to apply ILM policy to custom index coming from Openshift cluster log forwarding. As It create index with app-write and does not have any aliases.

GET app-write/_ilm/explain output

{
  "indices": {
    "app-write": {
      "index": "app-write",
      "managed": false
    }
  }
}

Any help would be appreciated.

3

Hi @Ofir_Edi i believe you tagged the wrong person.

4

Thanks, repositing.

Hi @lalchand_rajak ,
Have you tried using an index template ?

By doing so you can instruct elasticsearch that every time a new index is created with certain pattern it will be given settings and mappings as stated in the template.

For example, inex template for all indices starts with app-write will get ILM policy test-policy:

PUT _index_template/app_write_template
{
    "index_patterns": [
        "app-write*"
    ],
    "template": {
        "settings": {
            "index": {
                "lifecycle": {
                    "name": "test-policy"
                }
            }
        },
        "mappings": {},
    "composed_of": []
	}
}

also you can manually assign ILM policy to existing index using PUT settings

PUT app-write/_settings
{
   "index": {
                "lifecycle": {
                    "name": "test-policy"
                }
     }
}

Hopes this help,
Ofir

5

Hi,
I have tried the suggested the changes. but now getting the below error.

{
  "indices": {
    "app-write": {
      "index": "app-write",
      "managed": true,
      "policy": "app-write",
      "index_creation_date_millis": 1685791382524,
      "time_since_index_creation": "3.77d",
      "lifecycle_date_millis": 1685791382524,
      "age": "3.77d",
      "phase": "hot",
      "phase_time_millis": 1686116928468,
      "action": "rollover",
      "action_time_millis": 1686114592688,
      "step": "ERROR",
      "step_time_millis": 1686117528554,
      "failed_step": "check-rollover-ready",
      "is_auto_retryable_error": true,
      "failed_step_retry_count": 2,
      "step_info": {
        "type": "illegal_argument_exception",
        "reason": "setting [index.lifecycle.rollover_alias] for index [app-write] is empty or not defined"
      },
      "phase_execution": {
        "policy": "app-write",
        "phase_definition": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_size": "10gb",
              "max_primary_shard_size": "10gb",
              "max_age": "1d"
            }
          }
        },
        "version": 8,
        "modified_date_in_millis": 1686113759464
      }
    }
  }
}
6

Hi,
Now your index has ILM policy linked to it (as opposed to the initial state where it wasn't managed).
The error you see occurs because your ILM policy has rollover action in it but you index was not preconfigured for rollover.
here is what you can do:

  1. don't use rollover for your index and add date or any other name identifier for you index - you will beable to move it to warmer nodes over time and delete it eventually.
  2. bootsrap your index for rollover. Tutorial hereTutorial here.
  3. use data stream insted of index: link
7

Hi,

the issue is that OpenShift forward log to Elasticsearch and it create index as app-write only. How can i create ILM for such index.

8

I tried above 2 steps but does not work. The index require aliases. Once I added manually index get deleted but once deleted the index then again index does not have the aliases. how can I add aliases in template itself in below

PUT _index_template/app_write_template
{
    "index_patterns": [
        "app-write*"
    ],
    "template": {
        "settings": {
            "index": {
                "lifecycle": {
                    "name": "test-policy"
                }
            }
        },
        "mappings": {},
    "composed_of": []
	}
}
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.