Restored from snapshoots 2 master nodes but several indecies are in unassigned state

Elastic Stack Elasticsearch
1

We had 3 node cluster where all nodes were masters. We accidently deleted 2 master nodes. We restore them from snapshots. However we have several indecies in unassign state:|

externalkeyhandler-000005 4     p      STARTED       0  208b 100.68.100.6 node2
externalkeyhandler-000005 1     p      STARTED       0  208b 100.68.100.6 node2
externalkeyhandler-000005 2     p      UNASSIGNED
externalkeyhandler-000005 3     p      STARTED       0  208b 100.68.100.6 node2
externalkeyhandler-000005 0     p      STARTED       0  208b 100.68.100.6 node2

The description is:

  "primary" : true,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "CLUSTER_RECOVERED",
    "at" : "2022-12-28T11:05:45.500Z",
    "last_allocation_status" : "no_valid_shard_copy"
  },
  "can_allocate" : "no_valid_shard_copy",
  "allocate_explanation" : "cannot allocate because a previous copy of the primary shard existed but can no longer be found on the nodes in the cluster",

What should we do?

2

Which version of Elasticsearch are you using? What is in the Elasticsearch logs?

3

It was 7.13.2 and we try to upgarde to 7.17.1

4

Sorry. Forgot log:

[2022-12-28T15:43:19,823][DEBUG][o.e.a.s.TransportSearchAction] [node3] [_na_][production$syncissue][0]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[production$customruletemplate, production$characteristicslist, searchconfiguration$contextlocking, production$businessentity, production$contextmapper, production$priceevent, production$message, production$mergedruleaction, production$distributioninfo, production$businessrequest, production$stackablepromotiongroup, production$entityconfiguration, production$contextmapperindex, production$auxiliaryobjectmethod, production$genericelement, production$workflow, production$ruletemplate, production$qualificationcriterion, production$entitymethod, production$pricingitemtype, production$productofferingcategory, production$productofferingrelationship, production$uirulemetadata, production$mergedcontextmapper, production$mergedruletemplate, production$price, production$productspecification, production$pricequalificationrule, production$businessentitytype, production$auxiliarytype, production$elementarytypemethod, production$productofferinggroup, production$promotiongroup, production$pricingcharacteristic, production$variantgroup, production$pricepolicy, production$productofferingtermpolicy, production$mergedruleactionwithpopulatedrefs, production$operator, production$mergedcontextmapperindex, production$syncissue, production$rulecontextmodel, production$discount, production$genericentityspecification, production$metadata, production$category, production$auxiliaryobject, production$productspectype, production$priceversion, production$customcontextmapper, production$qualificationcriteria, production$propertytype, production$customcontextmapperindex, production$compositeprice, production$policy, production$mergedcontextwithnonpopulatedrefs, production$ruleskeleton, production$externalrecord, production$customoperator, production$elementarytype, production$ruleaction, production$pseudoattribute, production$customrulecontextmodel, production$event, production$pricegroup, production$productoffering, production$informationelement, production$mergedoperator, production$genericelemententry, production$buildingblock, production$metadatatype, production$businessrule, production$productofferingtype, production$mergedcontextwithpopulatedrefs, production$population_, production$pricegroupversion, production$promotion], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, expand_wildcards_hidden=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=5, batchedReduceSize=512, preFilterShardSize=null, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={"size":3,"query":{"match_all":{"boost":1.0}}}}] lastShard [true]
org.elasticsearch.action.NoShardAvailableActionException: null
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.performPhaseOnShard(AbstractSearchAsyncAction.java:292) [elasticsearch-7.13.2.jar:7.13.2]
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.