Auto-delete zipkin indices

Hi.
I am using Elastic Search 7.3.1 as a storage engine for OpenZipkin. Zipkin creates a span index daily in the format "zipkin-span-YYYY-MM-dd".

How I can automate the deletion of the Zipkin indices when its old than 10 days?

I gave a try to Elastic Search ILM with below PUT requests:

PUT http://localhost:9200/_ilm/policy/zipkin_indices_policy

    {
        "policy": {"phases": {"hot": {"actions": { "rollover": {"max_age": "03m"}}},
                                            "delete": {"min_age": "02m","actions": {"delete": {}}}}
                       }
    }

PUT http://localhost:9200/_template/zipkinidx_template

    {
       "index_patterns": ["zipkin*"],
       "settings": {"number_of_shards": 1, "number_of_replicas": 1,
                            "index.lifecycle.name": "zipkin_indices_policy"
                          }
    } 

but it fails at the action rollover with the below error:

    {"indices":{"zipkin-span-2020-09-04":{"index":"zipkin-span-2020-09-04","managed":true,"policy":"zipkin_indices_policy","lifecycle_date_millis":1599223883087,"phase":"hot","
   phase_time_millis":1599223884147,"action":"rollover","action_time_millis":1599223926386,"step":"ERROR","step_time_millis":1599224524930,"failed_step":"check-rollover-ready"
    ,"step_info":{"type":"illegal_argument_exception","reason":"setting [index.lifecycle.rollover_alias] for index [zipkin-span-2020-09-04] is empty or not defined","stack_trac
    e":"java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [zipkin-span-2020-09-04] is empty or not defined\n\tat org.elasticsearch.xpack.co
    re.indexlifecycle.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:50)\n\tat org.elasticsearch.xpack.indexlifecycle.IndexLifecycleRunner.runPeriodic
    Step(IndexLifecycleRunner.java:133)\n\tat org.elasticsearch.xpack.indexlifecycle.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:270)\n\tat org.elasticsear
    ch.xpack.indexlifecycle.IndexLifecycleService.triggered(IndexLifecycleService.java:213)\n\tat org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(Schedul
    erEngine.java:168)\n\tat org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:196)\n\tat java.base/java.util.concurrent.Executors$
    RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ScheduledThreadPoolEx
    ecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat j
    ava.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:835)\n"},"phase_execution":{"polic
    y":"zipkin_indices_policy","phase_definition":{"min_age":"0ms","actions":{"rollover":{"max_age":"3m"}}},"version":5,"modified_date_in_millis":1599223847778}}}}
  • Is it a must for an index to have an alias?
  • As the creation of the index is handled by the Zipkin and it does not create an alias for the index how can I proceed autodeletion of the index?

(I know one possible solution is to use Curator or hit DELETE API from a cron job but I am looking a solution from ilm perspective)

Thanks.

For ILM, yes.

Move from daily indices to letting ILM manage rollover for you.

Hi Mark. Thanks for the reply.

As adding an alias to the index is not in our control (index is created by Zipkin), I have to proceed with the curator at least for now.

Does ILM or the ILM's alias feature need any commercial license? If not, I will check with the openzipkin team if they can add alias to the indices.

The feature that requires an alias within ILM is rollover. This should as far as I know not be mandatory so you should be possible to use ILM with the indices exactly as they are currently created by zip kin.

Try taking out hot section of your ilm...

PUT http://localhost:9200/_ilm/policy/zipkin_indices_policy

{"policy": {"phases": {"delete": {"min_age": "02m","actions": {"delete": {}}}}}}

Hi Izukel. Many thanks for the reply which turned to the solution.

Yes, removing the hot phase and keeping only the delete phase in the policy removed the index.

Hi Christian. Thanks for the reply.

Ignoring the rollover phase from the policy and to consider only delete phase worked.