How works Allocation shards data tiers recommanded

Elastic Stack Elasticsearch
1

Hi everybody,
Fine ?

Questions about the allocation of the shards :wink:

I have a big index that has his dedicated index template with 3 primary shards and 1 replica

{
  "order": 1,
  "index_patterns": [
	"tdir_business_prod-*"
  ],
  "settings": {
		"index.lifecycle.name": "Hot_toWarm_toCold_policy",
		"index.refresh_interval": "60s",
	  "index.number_of_shards": 3,
	  "index.number_of_replicas": 1

		}
  }

The ILM policy with data tiers options recommanded

PUT _ilm/policy/Hot_toWarm_toCold_policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "2d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          },
          "set_priority": {
            "priority": 50
          }
        }
      },
      "cold": {
        "min_age": "6d",
        "actions": {
          "set_priority": {
            "priority": 25
          }
        }
      },
      "delete": {
        "min_age": "150d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          },
          "wait_for_snapshot": {
            "policy": "tdirprod_daily_snapshots_without_business"
          }
        }
      }
    }
  }
}

When I check the allocation of the shards of this index, all the shards are not on my 3 data COLD

index                         shard prirep state       docs  store ip          node
tdir_business_prod-2023.05.04 1     p      STARTED 36909044 38.6gb 172.20.0.30 node_data_tlragsa030.a2569
tdir_business_prod-2023.05.04 1     r      STARTED 36909044 38.6gb 172.20.0.55 node_data_cold_tlragsa055.a2569
tdir_business_prod-2023.05.04 2     r      STARTED 36895180 38.6gb 172.20.0.31 node_data_tlragsa031.a2569
tdir_business_prod-2023.05.04 2     p      STARTED 36895180 38.6gb 172.20.0.51 node_data_tlragsa051.a2569
tdir_business_prod-2023.05.04 0     p      STARTED 36844203 38.5gb 172.20.0.40 node_data_cold_tlragsa040.a2569
tdir_business_prod-2023.05.04 0     r      STARTED 36844203 38.5gb 172.20.0.39 node_data_cold_tlragsa039.a2569

p0, r0, r1 are on COLD DATA and p1,p2,r2 are on HOT DATA
Why p1,p2,r2 are not on COLD DATA and why if there is a bug, there are not on WARM DATA instead ?

I need to free up disk space on HOT DATA and WARM DATA to have big disk space on COLD DATA
Does i have to use node.attribute to ensure the allocation in WARM and COLD DATA and shrink the index in warm state ?

The allocation works well with the others indices that have only 1 primary shard and 1 replica

thanks a lot

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.