How do index rollover and tier transition work when they overlap

Elastic Stack Elasticsearch
,
1

Hi,
I'm a little confused on how transitioning a data stream's write index to the warm phase would work if it's still not supposed to be rolled-over yet.

Here's an example. Let's say I have an ILM to rollover the data stream indices when they reach an absurd 500 GBs, but also the policy is set to transition the index to the warm tier when the index is just 1 day old.

Now what happens when the index is just 10 GBs in size, but 2 days old? Is it rolled-over? Is it moved to the warm phase even though it's the data stream's write index?

In generic terms, what happens when the data stream's write index hasn't met the rollover criteria but it did meet the phase transitioning criteria?

Is this mentioned in the docs anywhere? I couldn't find it.

Thanks!

2

Hi @EyadArafat Welcome to the community!

Good Question!

Rollover is the first Phase.

Transition to the warm / next phase is calculated from Rollover...

See Here

A lifecycle policy specifies the phases in the index lifecycle and the actions to perform in each phase. A lifecycle can have up to five phases: hot, warm, cold, frozen, and delete.

For example, you might define a timeseries_policy that has two phases:

  • A hot phase that defines a rollover action to specify that an index rolls over when it reaches either a max_primary_shard_size of 50 gigabytes or a max_age of 30 days.
  • A delete phase that sets min_age to remove the index 90 days after rollover.

The min_age value is relative to the rollover time, not the index creation time.

So say your rollover takes 30 days BUT you have Warm at 7 Days... Then Warm would happen at 37 Days :slight_smile:

There are some detailed setting to change that behavior but if you are just getting started I would understand this first and work from there.

1 Like
3

Ah I see. That makes sense. Thanks!

So that means I can set the transition to warm to be done on '0 days' to have the index move straight to the warm tier once it gets rolled-over, right?

That way I can only keep the write-able indices in the hot tier and all the read-only indices in the warm tier.

4

Correct... Just FYI, It Will not happen instantly. It will happen soon after. Just letting you know some people expect it to happen within milliseconds. ILM is a background task that runs on about a 10 minute interval or so.

5

Got it. That clears everything up, thank you!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.