Restore not working on rollover Indices

Elastic Stack Elasticsearch
,
1

I have two indices in my setup.(below indices are rolledover indices using ILM)

1."test_syslogs_1-000002" : {
   "aliases" : {
     "test_syslogs_1" : {
       "is_write_index" : false
     }
   }
}
2. "test_syslogs_1-000003" : {
   "aliases" : {
     "test_syslogs_1" : {
       "is_write_index" : true
     }
   }
}

When I tried to restore the indices from the snapshot using

{
  "indices": "test_*",
  "ignore_unavailable": true,
  "include_global_state": true,
  "rename_pattern": "test_(.+)",
  "rename_replacement": "restored_test_$1"
}

But this restore is throwing below exception

[2020-05-12T14:53:49,036][WARN ][o.e.s.RestoreService     ] [es02] [snapshot-1/4_ytokhfRymOzcjvI_0_MQ] failed to restore snapshot
java.lang.IllegalStateException: alias [test_syslogs_1] has more than one write index [restored_test_syslogs_1-000002,test_syslogs_1-000003]
        at org.elasticsearch.cluster.metadata.AliasOrIndex$Alias.computeAndValidateWriteIndex(AliasOrIndex.java:154) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.metadata.MetaData$Builder.lambda$buildAliasAndIndexLookup$2(MetaData.java:1295) ~[elasticsearch-7.6.2.jar:7.6.2]
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?]
        at java.util.TreeMap$ValueSpliterator.forEachRemaining(TreeMap.java:2890) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
        at org.elasticsearch.cluster.metadata.MetaData$Builder.buildAliasAndIndexLookup(MetaData.java:1295) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1258) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.ClusterState$Builder.metaData(ClusterState.java:703) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.snapshots.RestoreService$1.execute(RestoreService.java:415) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:702) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:324) ~[elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:219) [elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:73) [elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) [elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:633) [elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-7.6.2.jar:7.6.2]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-7.6.2.jar:7.6.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]

Issue: Even though both are not write indices for same alias,it is giving above exception.
Please help me with this issue.

One reason could be, when the snapshot is taken 000002 indices is having write aliases and now 000003 index is having write aliases.

2

It looks like there was an illegal state when the data was snapshotted. Can you try restoring the snapshot with the include_aliases: false parameter in your restore body and see if you're able to restore the snapshot then?

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.