Hi,
A few days ago, I've just installed a new node to my single node cluster, in order to manage datastreams lifecycle policies between 2 nodes :
- elk1 is configured in elasticsearch.yml with
node.roles: master, data_content, data_hot - elk2 is configured in elasticsearch.yml with
node.roles: data_warm, data_cold, data_frozen
There is no replica node in the cluster.
Since, I have some 'unassigned shards' created each time a datastream need to create a new indice.
GET _cluster/allocation/explain
{
"index": ".ds-logs-security-default-2023.06.20-000004",
"shard": 0,
"primary": false
}
{
"index": ".ds-logs-security-default-2023.06.20-000004",
"shard": 0,
"primary": false,
"current_state": "unassigned",
"unassigned_info": {
"reason": "INDEX_CREATED",
"at": "2023-06-20T01:06:45.736Z",
"last_allocation_status": "no_attempt"
},
"can_allocate": "no",
"allocate_explanation": "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there.",
"node_allocation_decisions": [
{
"node_id": "QkN3guLiS5qMZBzZpMsBoA",
"node_name": "elkglbvprd2",
"transport_address": "172.18.0.219:9300",
"node_attributes": {
"xpack.installed": "true"
},
"node_decision": "no",
"weight_ranking": 1,
"deciders": [
{
"decider": "data_tier",
"decision": "NO",
"explanation": "index has a preference for tiers [data_hot] and node does not meet the required [data_hot] tier"
}
]
},
{
"node_id": "AjJt4OmLSdWLuCzGPK_bpQ",
"node_name": "elkglbvprd1",
"transport_address": "172.18.0.218:9300",
"node_attributes": {
"xpack.installed": "true"
},
"node_decision": "no",
"weight_ranking": 2,
"deciders": [
{
"decider": "same_shard",
"decision": "NO",
"explanation": "a copy of this shard is already allocated to this node [[.ds-logs-security-default-2023.06.20-000004][0], node[AjJt4OmLSdWLuCzGPK_bpQ], [P], s[STARTED], a[id=HALmKeVyTni1ON1M56IG-g], failed_attempts[0]]"
}
]
}
]
}
What I understand is that the new indice is created on node elk1 but cannot be created on node elk2 because this node doesn't have [data_hot].
I think it's because the new indice is created with a default replica count of 1, as when I set the number of replica too 0, error is diseappering.
So, is there a way to configure datastreams to have 0 in number of replica to avoid this error each time a new indice is created for the existing datastream?
Plus, is there a way to have this configured by default to all new datastreams ?
Regards.