Unable to create primary shard in hot-1 and replica shard in hot-2

[aris_lawrence (Aris)]

Elasticsearch version 7.7.1

We have the following setup in our elastic environment.
3 master only
2 ingest only
2 hot nodes (hot-1-p1 and hot-2-p1)
2 warm nodes (warm-1-p1 and warm-2-p1)
1 cold node (cold-p1)

master node configuration
routing:
  allocation:
    enable: all
    awareness.attributes: rack_id
    disk.watermark.low: 80%
    disk.watermark.high: 85%
    disk.watermark.flood_stage: 88%

hot node configuration
node:
  name: hot-1-p1 #hot-2-p1 for the second 
  attr.rack: hot
  attr.rack_id: hot
  attr.box_type: hot
  attr.data: hot
  master: false
  data: true
  ingest: false

warm node configuration
node:
  name: warm-1-p1 #warm-2-p1 for the second 
  attr.rack: warm
  attr.rack_id: warm
  attr.box_type: warm
  attr.data: warm
  master: false
  data: true
  ingest: false

cold node configuration
node:
  name: cold-p1
  attr.rack: cold
  attr.rack_id: cold
  attr.box_type: cold
  attr.data: cold
  master: false
  data: true
  ingest: false

index template
setup.template.settings:
  index.number_of_shards:1
  index.number_of_replicas: 1

I am trying to create 1 primary and 1 replica shard but the replica is being created in warm node. Is it possible to create the 1 primary to "hot-1 or hot-2" and the replica to "hot-2 or hot-1"?

[warkolm (Mark Walkom)]

You need to add the filtering attribution to the index as well, check out https://www.elastic.co/guide/en/elasticsearch/reference/7.10/shard-allocation-filtering.html

[aris_lawrence (Aris)]

I've added

This is the result

[warkolm (Mark Walkom)]

What does https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-allocation-explain.html show?

[aris_lawrence (Aris)]

Sorry but i can only upload images since copy/paste is disabled between my computer and the server.

EDITED: I may need to remove some of the attr in the yml file and experiment with the filtering.

[aris_lawrence (Aris)]

Ok, i finally figured it out. It was the awareness that was causing the trouble and "maybe" the other attributes.

On the master nodes elasticsearch.yml, i removed "routing.allocation.awareness.attributes".
On the data nodes (hot/warm/cold), i only retained "node.attr.data"
On the index templates, i changed it to "index.routing.allocation.require.data: hot"

Thanks for the assistance.