Is it possible to create rolling index using template?

Elastic Stack Elasticsearch
1

Like the title suggests.
The process of creating a rolling index is to create "indexName-000001" with alias "indexName".

The way to use rolling index is to have the applications write data to "indexName" (the alias), and periodically call the _rollover api with parameters.

To me, it sounds impossible with today's template structure; therefore, I have been creating rolling index manually even though we use template for other hard indices. Is my assumption incorrect?

Thanks.

2

Hi @linkerc
What version are you on?
What you are trying to do is definitely supported with the correct templates, write alias or data stream and an Index Lifecycle Policy (ILM).

3

8.10.2.
We don't use ILM. We manage the rollover and deletion ourselves.

We are simply using rollover index to support huge documents. It's not meant to move data into "phases".

ILM seems to be designed for folks who don't care about fine control of data retention. I tried to read up on it before and it seems complicated. Especially with template. Do I have to create a data stream for what I'm trying to do?

4

What you described is exactly the exactly the use case for ILM.

ILM is used to help you manage the lifecycle of your indices and follow the recommendations regarding indices, like shard sizes and max number of documents per shard.

You can use ILM just to rollover your indices, there is no obligation to move them between different data tiers, you can also use with normal indices and not just data streams.

Not sure what you mean with fine control of data retention, but this is another thing where ILM can help you.

What are your requirements for rollover? When using ILM you can configure the indices to rollover by age, by size or by number of documents, you can also force a rollover if you need.

This documentation explains how to use automate rollover for indices without data streams: Tutorial: Automate rollover with ILM | Elasticsearch Guide [8.13] | Elastic

5

Thanks for the link. I could be more cleared with my original wording.
Bootstrapping the initial index-000001 is still required even with template.

My question was meant to ask if template can fully automate the whole rolling index without any intervention.

The reason I ask this is we create index by appending month to the end of index name based on the payload's timestamp. So our data is grouped monthly per index per se. It works great with template when it's not rolling index because there's no seeding/bootstrapping.

Template simply works. The writers just need to insert data, etc.
But if I need rolling index, then code/human intervention for bootstrapping will be required.

Is my understanding correct?