We have recently added node attributes and update the cluster setting to allocate the shards based on this attribute. below is my cluster settings.
"cluster" : {
"routing" : {
"allocation" : {
"awareness" : {
"attributes" : "rack"
}
}
}
},
though after a while we have found cluster health, yellow, and when checking shards, figured out that .security-7 index has unassigned shards.
the result of below query is attached:
GET _cluster/allocation/explain
{
"index": ".security-7",
"primary": false,
"shard": 0
}
and the result is:
"node_allocation_decisions" : [
{
"node_id" : "0yD-okm5StKqAUyO0zFjkA",
"node_name" : "elasticsearch-warm-ebs115-1",
"transport_address" : "10.220.64.230:9300",
"node_attributes" : {
"rack" : "SRV-115",
"xpack.installed" : "true",
"transform.node" : "false"
},
"node_decision" : "no",
"deciders" : [
{
"decider" : "awareness",
"decision" : "NO",
"explanation" : "there are [6] copies of this shard and [16] values for attribute [rack] ([SRV-101, SRV-102, SRV-103, SRV-104, SRV-105, SRV-106, SRV-107, SRV-108, SRV-109, SRV-110, SRV-113, SRV-114, SRV-115, SRV-14, SRV-15, SRV-16] from nodes in the cluster and no forced awareness) so there may be at most [1] copies of this shard allocated to nodes with each value, but (including this copy) there would be [2] copies allocated to nodes with [node.attr.rack: SRV-115]"
},
{
"decider" : "data_tier",
"decision" : "NO",
"explanation" : "index has a preference for tiers [data_content] and node does not meet the required [data_content] tier"
}
]
}, ...
I know the problem with shard allocation and the number of my distinct racks, though I want to know how can I reduce the number of such an index replication, or ignore the allocation awareness while it is not possible. since I have not force allocate.
this is the .security index settings:
#! this request accesses system indices: [.security-7], but in a future major version, direct access to system indices will be prevented by default
{
".security-7" : {
"settings" : {
"index" : {
"refresh_interval" : "1s",
"auto_expand_replicas" : "0-5",
"provided_name" : ".security-7",
"format" : "6",
"creation_date" : "1662378412854",
"unassigned" : {
"node_left" : {
"delayed_timeout" : "10m"
}
},
"analysis" : {
"filter" : {
"email" : {
"type" : "pattern_capture",
"preserve_original" : "true",
"patterns" : [
"([^@]+)",
"""(\p{L}+)""",
"""(\d+)""",
"@(.+)"
]
}
},
"analyzer" : {
"email" : {
"filter" : [
"email",
"lowercase",
"unique"
],
"tokenizer" : "uax_url_email"
}
}
},
"priority" : "1000",
"number_of_replicas" : "5",
"uuid" : "XZvygi9aRMmskeM0fVF3qQ",
"version" : {
"created" : "7160399"
},
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1"
}
}
}
}