Index ILM+logstash

Elastic Search
1

Hi i try use logstash for my data with ILM policy.
Index rollover and create new index with template and so on. But the name of the index just increase a number but not wroks with date. Date is still the same.

Here is a part of logstash output for this index

manage_template => false
ilm_rollover_alias => huawei_sw
ilm_pattern => "{now/d}-000001"
ilm_policy => Huawei_SW_ILM_policy

Can u help where is the problem?

2
  1. Hows your policy looks like? If you can share?
  2. Is your first index is create with date prefix?
3

hi,

ILM policy:

    "version": 9,
    "modified_date": "2024-11-19T07:19:44.783Z",
    "policy": {
      "phases": {
        "cold": {
          "min_age": "5h",
          "actions": {
            "allocate": {
              "include": {
                "_tier_preference": "data_cold"
              },
              "exclude": {},
              "require": {}
            },
            "set_priority": {
              "priority": 0
            }
          }
        },
        "delete": {
          "min_age": "365d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_age": "5h",
              "max_docs": 1000,
              "max_size": "5gb"
            },
            "set_priority": {
              "priority": 100
            }
          }
        }
      }
    },

and yes i first create ILM policy template a then first index

PUT huawei_sw-2024.11.19-000001
{
  "aliases": {
    "huawei_sw": {
      "is_write_index": true
    }
  }
}
4

You have to use the "date math" when you create the first index, see API conventions | Elasticsearch Guide [8.16] | Elastic.

You could use data streams, they are easier.

5

so this shoul be work yes?

PUT /%3Chuawi_sw-index-%7Bnow%2Fd%7D%3E
{
  "aliases": {
    "huawei_sw": {
      "is_write_index": true
    }
  }
}
6

Looks better, but I'm not going to desk check it here :slight_smile:

Create the index, look at it in stack management (or GET), the setting provided_name needs to contain "-{now/d}-". If you look at your first one, it has the literal date.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.