Is there a way to log or to check the log with time taken to complete the transition from Phases in ILM

[pruthvi]

I have implemented ILM(index lifecycle management) with roller over from Hot to warm and move to cold after period of time.
I have indexed few thousand of document into index. Rollover is also happing with max age in policy.

Question: I wanted to log or check the log for the IndexLifecycleTransition with time taken to complete the transition from Phases.

I could only see below log in elastic-cluster-primary.log mentioning only 'from' 'to' details without any details on time it took to perform the Transition.

[2023-01-31T14:22:27,724][INFO ][o.e.x.i.IndexLifecycleTransition] moving index [index-2023.01.30-000020] from [{"phase":"warm","action":"complete","name":"complete"}] to [{"phase":"cold","action":"migrate","name":"migrate"}] in policy [index-policy-2]
[2023-01-31T14:22:28,507][INFO ][o.e.x.i.IndexLifecycleTransition] moving index [index-2023.01.30-000020] from [{"phase":"cold","action":"migrate","name":"migrate"}] to [{"phase":"cold","action":"migrate","name":"check-migration"}] in policy [index-policy-2]
[2023-01-31T14:22:31,777][INFO ][o.e.x.i.IndexLifecycleTransition] moving index [index-2023.01.30-000020] from [{"phase":"cold","action":"migrate","name":"check-migration"}] to [{"phase":"cold","action":"complete","name":"complete"}] in policy [index-policy-2]

[stephenb (Stephen Brown)]

Hi @pruthvi Welcome to the community

@pruthvi

I could only see below log in elastic-cluster-primary.log mentioning only 'from' 'to' details without any details on time it took to perform the Transition.

Curious about what you are trying to figure out. At Scale ILM is opportunistic in the background and depending on the transition phases there are different processes that will be performed, so it will not always be the same...

Moving from Hot to Warm is different from Warm to Cold or from Cold to Frozen.
Also, the Data is not removed from the Previous tier until the next tier is valid.

if you run this command you will get some more details

GET index-2023.01.30-000020/_ilm/explain

[pruthvi]

Hi @stephenb,Thank you.
I was trying to figure out performance or time it takes while ILM perform the Transition between phases and log it. So I may have some idea over Transition time with similar size data as in production.