Using ILM rollover with multiple indices

Elastic Stack Elasticsearch
1

Enviroment

ES: 7.8.1
Kibana: 7.8.1
Run in docker: 19.03.8, build afacb8b
Using docker image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1

Hi everybody,
I'm having a hard time with using ILM rollover with multiple indices.
For example, I have tree indices.

  • index-xxx
  • index-yyy
  • index-zzz
    and they have same alias: index-all

The index template is:

  • index pattern: index-*
  • Setting
{
  "index": {
    "lifecycle": {
      "name": "index-ilm",
      "rollover_alias": "index-all"
    }
  }
}
  • Alias
{
  "index-all": {}
}

The ILM rollover is enabled.
The problem is, when the ILM start, it tells me that
illegal_argument_exception: source alias [index-all] does not point to a write index
I read the doc and the doc tells me that I should set is_write_index: true for one of my index.
But I have multiple indices and ES only allow me to set one of my index, how can I deal with it.

Purpose

I want all my index can use rollover using same alias like:

  • index-xxx-00001
  • index-yyy-00001
  • index-zzz -00001
  • index-xxx-00002
  • index-yyy-00002
  • index-zzz -00002
  • shrink-index-xxx
  • shrink-index-yyy-00001
  • shrink-index-zzz -00001

Another Problem

When I disable the rollover , it follow each stage min_age, but when a index go into shrink/cold stage(shrink-index-zzz), I try to insert a new doc in index shrink-index-zzz, it will not auto create a new index and return 403 Forbiddent.

Thanks for helping!

2

If you want to use rollover I believe you need a policy per index name patter as they need to have different write aliases.

3

yearh... I agree. Is there have some better solution?

4

Why not use standard time based indices, but change from daily to weekly or monthly. If an index gets too large you can always split it later.

5

That is a good way to do it but I still want to use the rollover function. :slight_smile:

6

Then i believe you need to find a way to create/manage a policy per index pattern. :slight_smile:

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.