Assigning failing SHARD's

[thulsidosskrishnan]

Hello Community,

Greetings. I have been trying to get few shards assigned so that the cluster comes

back to green status.

I have been trying to understand the impact of disable alloaction .

Could you please validate these steps , so that i can know if these are the correct

steps to follow to put the cluster back into green mode.

There were few CircuitBreakingException on Dec7th and there were attempts

to further check on the cluster state. Here is what followed

1.Checking the clsuter state :cluster is in yellow ,few shards are failing

2.the failing shards are queried to find the actual reason using the allocation API

call,the exception thrown matches the issue with the circuit breaker exception

obtained on the Dec7th.

3.The cluster PODS were increased in heap size to over come the circuit breaker

exception issue.

4.After Dec7th the cluster failing shards were analyzed to find the issue for

unallocation.

5.So as per the article
medium link

Elastic search wont assign

an unassigned SHARD after max retry.

To do the unassigned shard allocation, here are the steps

1. Disable allocation
2. Reset the retry counter with a bare POST /_cluster/reroute?retry_failed=true
3. Execute the desired reroute command with another call to POST /_cluster/reroute
4. Re-enable allocation

Please validate if the above steps are indeed to be followed to try the unassigned shard allocation.

Another question is what is the impact of "Disable allocation" especially in a

PROD cluster ie the effect of running this POST command . Does that mean newer

SHARDs creation will be momentarily disabled.

PUT _cluster/settings
{
"persistent": {
"cluster.routing.allocation.enable": "none"
}
}

Also has the reassignemnt of each shard has to be done on one-by-one basis.

Please also clarify when the "reroute" API is executed does that read the whole

request again (saved in some state in the form of failed SHARD) .

If that request is read again then possibly a newer POD heap size is required to

support all the failed SHARDs reassignment ?

Thanks in advance
Thulsi Doss Krishnan