Dynamic rollover alias and template name

Hi.

Could it be possible to allow using variables in the properties " ilm_rollover_alias " as well as " template_name " in the Elasticsearch output, like we can do with the "index" property?

Index lifecycle error
illegal_argument_exception: index.lifecycle.rollover_alias [server-log-*-] does not point to index [server-log-demoproduct-2021.08.03]
Stack trace

My indices is "server-log-%{product}-%{+YYYY.MM.dd}"

In the output of what sorry?

Hi. This is error in kibana
java.lang.IllegalArgumentException: index.lifecycle.rollover_alias [server-log-*-] does not point to index [server-log-demovn-2021.08.03]
at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:124)
at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:175)
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:335)
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:273)
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:184)
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:217)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:831)

My template setting
{
"index": {
"lifecycle": {
"name": "server-log-policy",
"rollover_alias": "server-log-*-"
},
"refresh_interval": "5s",
"number_of_shards": "1",
"number_of_replicas": "0"
}
}

logstash output

elasticsearch {
hosts => ["http://10.10.10.8:9200"]
#ilm_enabled => "true"
#ilm_pattern => "000001"
#ilm_policy => "server-log-policy"
#ilm_rollover_alias => "server-log"
#manage_template => false
user => "elastic"
password => "xxx"
index => "server-log-%{product}-%{+YYYY.MM.dd}"
}

Ok that makes more sense, you mean Logstash not Elasticsearch :slight_smile:
When you send data from Logstash to an ILM policy in Elasticsearch, you need to send it to the write alias and not the underlying index pattern.

Just a quick ask for the future - please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile:

Thank for your reply. I have changed configure in logstash to send data to the write alias.

elasticsearch {
                        hosts => ["http://10.10.10.8:9200"]
                        ilm_enabled => "true"
                        #ilm_pattern => "000001"
                        ilm_policy => "server-log-policy"
                        ilm_rollover_alias => "server-log-%{product}"
                        #manage_template => false
                        user => "elastic"
                        password => "1DoQN06dmFOIdU3ZZhPJ"
                        #index => "server-log-%{gameid}-%{+YYYY.MM.dd}"
                }

But logstash is failed

[2021-08-04T08:12:33,255][ERROR][logstash.outputs.elasticsearch][log] Failed to install template. {:message=>"Malformed escape pair at index 23: /_template/server-log-%{product}"

I found a similar problem here Impossible to use variables in ilm_rollover_alias · Issue #10962 · elastic/logstash · GitHub

Correct, you cannot use variables there unfortunately.

Thank for your reply.
I will disable rollover option in ilm policy And manage rollover by the underlying index pattern in logstash configuration.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.