Hi All,
I have the following structure of indices, that are created as needed, based on a template, to store documents received from a different regions:
some-data-from-region-us-000001
some-data-from-region-eu-000001
.....
The region ID is configurable, so I don't have it beforehand. The template is configured/linked to an ILM-Policy.
During index creation, each regional index gets it's own regional alias: (e.g. some-data-from-region-eu-alias), which is used to write data to this index.
That means, I have multiple Write-Indices (active) but each with a different regional aliases.
But this conflicts with the ILM-Feature, as I can configured only ONE alias. Hence I'm getting the following error:
index.lifecycle.rollover_alias [some-data-from-region] does not point to index [some-data-from-region-us-000001]
Does this mean, I have to maintain an ILM-Policy & Template for each regional index to use ILM? Or perhaps it's possible to update the indices rollover-alias after the index have been created. (e.g. using an interval, that validates the alias)
I'm asking, to make sure, that there is no other option I have overlooked.
I think it's related to this GitHub issue: https://github.com/elastic/elasticsearch/issues/38454, which has been closed in favor to Data-Streams. Unfortunately I cannot use Data Streams because upserts are not supported to my knowledge. I'm using Logstash to send documents.
If someone has some tips or ideas how to handle this use-case or just share best/practices/recommendations I would appreciate it.
Thanks!
Chris