ILM Questions

[Wayne_Taylor]

Hi Elastic Team,

I know there is a lot of questions on ILM which encompasses both the Elasticsearch aspects as well as Kibana.

Background:
We use Fluentd for our log ingestion framework developed by our DevOps teams that takes logs from Docker containers and pushes them into Elasticsearch. This process is working well. Its highly available, its fault tolerant, etc. Our index naming patterns are like ec2.<environment>.<region>.<appname>.

Our initial Elastic Cloud setup wasn't configured with Hot-Warm architecture. But i've recently added Dense Storage nodes and ensured that they are not used for data ingestion.

We are currently working with our elastic rep for a 1 year commit with a 3 month retention (7 days hot or 50GB, warm for remaining, followed by delete afterwards). We currently don't have support today

Help I need to understand:
If i create an ILM Policy that has 7 days hot OR 50GB -> Move to warm enabled with it pointing to warm nodes followed by delete after X days. I get errors.

First set of errors and steps below:

1. After policy is created. I then try to associate it to existing indexes. When I do so i see: Policy ec2_build is configured for rollover, but index ec2.build.test does not have an alias, which is required for rollover.
2. I then create an alias:
   POST /_aliases{ "actions" : [{ "add" : { "index" : "ec2.build.*", "alias" : "ec2_build_alias", "is_write_index" : true} }]}
3. I then associate this this alias to the index template:
   PUT _template/ilm_build{ "order": 10,"index_patterns": ["ec2.build.*"],"settings": {"index.lifecycle.name": "myilmpolicy", "index.lifecycle.rollover_alias": "ec2_build_alias" }}
4. I then get errors: alias [ec2_build_alias] has more than one write index

Am I missing some pre-setup somewhere. If so, can someone outline specifics. I seem to be missing clearly in docs or isn't clear.

Thanks
Wayne

[Stef_Nestor]

(Apologies for the original lack of response.)

Appending for when searched in the future, an index can have an "alias" and a "rollover_alias". This was erring on not having the latter. For a setup example similar to above with error recovery examples, see our new blog on Troubleshooting ILM.

Copy pasting for this case, Index Settings would look like

GET INDEX_NAME-000001/_settings?filter_path=*.settings.index.lifecycle
{
    "INDEX_NAME-000001" : {
        "settings" : {
            "index" : {
                "lifecycle" : {
                    "name" : "INDEX_POLICY_NAME",
                    "rollover_alias" : "INDEX_ALIAS"
                }
            }
        }
    }
}

And second error would recover

alias [x] has more than one write index [y,z]

When you run Get Aliases, you’ll notice that two indices are marked as is_write_index:true when only one should be per alias. You’ll want to toggle is_write_index:false on one of the indices via the Aliases API.