Enrich Indices have unassigned shards

Elastic Stack Elasticsearch
1

Hello,

I understand why Elastic manages the primary and replica shards of enrich indices. I get that the whole point is parallelism, but the end user should be able to manage the shard settings. It is causing an imbalance because the elastic logic is applied.

Currently, the unassigned shards (all coming from enrich indices) are not being assigned because the nodes are reporting in _cluster/allocation/explain that every node already has a shard:

"explanation": "a copy of this shard is already allocated to this node [[.enrich-<redacted>][0], node[<redacted>], [R], s[STARTED], a[id=<redacted>], failed_attempts[0]]"

This are the default settings:

image
image700×282 12.3 KB

image
image337×106 2.29 KB

I ran this:

PUT .enrich-*/_settings {"index": {"number_of_replicas": 1}}

And did nothing because it’s not manageable by the end user.

ES version: 9.1.3

2

Can you share the full allocation explain?

3

I can’t, but I believe I figured it out. In the cluster allocation explanation, one hot node has hit the watermark for disk usage. It is not allowing new shard allocation to that node. Therefore, I can assume that because there’s a copy of that shard everywhere else but the node, it wants to assign the replica shard but can’t. Leaving unassigned replica shards for enrich indices.

4

I fixed the issue with the hot node's lack of disk space for replicas; now, the cluster is green.

The underlying issue is storage constraints, but I feel like Elastic should allow the user to manage the enrich indices’ replica shard settings. It's overdoing it by creating so many replica shards.