ILM Questions

Hi Elastic Team,

I know there is a lot of questions on ILM which encompasses both the Elasticsearch aspects as well as Kibana.

Background:
We use Fluentd for our log ingestion framework developed by our DevOps teams that takes logs from Docker containers and pushes them into Elasticsearch. This process is working well. Its highly available, its fault tolerant, etc. Our index naming patterns are like ec2.<environment>.<region>.<appname>.

Our initial Elastic Cloud setup wasn't configured with Hot-Warm architecture. But i've recently added Dense Storage nodes and ensured that they are not used for data ingestion.

We are currently working with our elastic rep for a 1 year commit with a 3 month retention (7 days hot or 50GB, warm for remaining, followed by delete afterwards). We currently don't have support today

Help I need to understand:
If i create an ILM Policy that has 7 days hot OR 50GB -> Move to warm enabled with it pointing to warm nodes followed by delete after X days. I get errors.

First set of errors and steps below:

  1. After policy is created. I then try to associate it to existing indexes. When I do so i see: Policy ec2_build is configured for rollover, but index ec2.build.test does not have an alias, which is required for rollover.
  2. I then create an alias:
    POST /_aliases{ "actions" : [{ "add" : { "index" : "ec2.build.*", "alias" : "ec2_build_alias", "is_write_index" : true} }]}
  3. I then associate this this alias to the index template:
    PUT _template/ilm_build{ "order": 10,"index_patterns": ["ec2.build.*"],"settings": {"index.lifecycle.name": "myilmpolicy", "index.lifecycle.rollover_alias": "ec2_build_alias" }}
  4. I then get errors: alias [ec2_build_alias] has more than one write index

Am I missing some pre-setup somewhere. If so, can someone outline specifics. I seem to be missing clearly in docs or isn't clear.

Thanks
Wayne

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.