Adding warm node to existing cluster, how to avoid existing indices to spread to these nodes

Elastic Stack Elasticsearch
#1

I plan to add 3 warm nodes into my cluster. (version 7.9 and 15 hot nodes)
I modified elasticsearch.yml with these new lines:

node.attr.data: hot # on existing nodes
node.attr.data: warm # on new nodes

when I start warm nodes, the cluster does not distinguish between warm/hot attr.data and indices are spread among all nodes.

How do I configure the cluster, that only ILM specific indices are copied to warm nodes. ?

it is not clear from https://www.elastic.co/blog/implementing-hot-warm-cold-in-elasticsearch-with-index-lifecycle-management

based on

should I configure on all nodes

cluster.routing.allocation.awareness.attributes: data

it this correct approach?

#2

I found i some examples I could set in templates/indices

  "settings": {
    "index": {
      "routing.allocation.require.data": "hot"

is this an better approach than ?

cluster.routing.allocation.awareness.attributes: data

Do I have to setup for all existing indices this settings?
Can use cluster setup?

doing it template/index way it makes the management of hot/warm/cold quite complex.

I am looking for some simple how-to
with example config setup.

It seems I have to create a template for * all indices
with setup of "index.routing.allocation.require.data": "hot"

or I have to modify default setting for indices if exists

for alle existsing cluster indices I have to configure this setting
"index.routing.allocation.require.data": "hot"
before I will add warm nodes

thank u

#3

That is the best approach.

Yes you need to tag all nodes appropriately.

1 Like
#4

Thank you
Is there a way I can configure cluster that all indices created will have "index.routing.allocation.require.data": "hot" without need of template.

so any index commands like this goes to hot nodes without need of template

POST my-index/_doc
{
"hello": "world"
}

This is my theoretical strategy to add warm nodes:

  1. update all nodes elasticsearch.yml
    node.attr.data: hot

  2. update all indices to have setting
    { "settings": { "index.routing.allocation.require.data": "hot" }

  3. update all templates to have setting
    { "settings": { "index.routing.allocation.require.data": "hot" }

  4. add warm nodes

(I am looking for default config, so all indices created will have this "index.routing.allocation.require.data": "hot" without need of template )
#5

The only other option is to do it manually.

1 Like
#6

do I have to modify also system templates like "ilm-history"
or I should keep this as is.