A recent outage required a restart of Elastic server.
The server seems to be doing fine for all other indexes (indexes are still being created) but the Kibana index was not able to recover. Elastic is throwing exception below.
Does anybody know how to best resolve this?
[[.kibana][0]] marking and sending shard failed due to [failed recovery]
[.kibana][[.kibana][0]] IndexShardRecoveryException[failed to recovery from gateway]; nested: EngineCreationFailureException[failed to create engine]; nested: TranslogCorruptedException[expected shard UUID [[76 54 38 7a 48 6f 47 6a 52 43 43 45 68 63 51 4f 57 6a 54 4d 78 67]] but got: [[4e 57 57 77 78 6f 56 38 51 4f 65 5a 7a 69 52 48 36 36 6e 66 57 41]] this translog file belongs to a different translog];
at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:250)
at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:56)
at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: [.kibana][[.kibana][0]] EngineCreationFailureException[failed to create engine]; nested: TranslogCorruptedException[expected shard UUID [[76 54 38 7a 48 6f 47 6a 52 43 43 45 68 63 51 4f 57 6a 54 4d 78 67]] but got: [[4e 57 57 77 78 6f 56 38 51 4f 65 5a 7a 69 52 48 36 36 6e 66 57 41]] this translog file belongs to a different translog];
at org.elasticsearch.index.engine.InternalEngine.(InternalEngine.java:155)
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1509)
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1493)
at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:966)
at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:938)
at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:241)
... 5 more
Caused by: TranslogCorruptedException[expected shard UUID [[76 54 38 7a 48 6f 47 6a 52 43 43 45 68 63 51 4f 57 6a 54 4d 78 67]] but got: [[4e 57 57 77 78 6f 56 38 51 4f 65 5a 7a 69 52 48 36 36 6e 66 57 41]] this translog file belongs to a different translog]
at org.elasticsearch.index.translog.TranslogReader.open(TranslogReader.java:235)
at org.elasticsearch.index.translog.Translog.openReader(Translog.java:377)
at org.elasticsearch.index.translog.Translog.recoverFromFiles(Translog.java:334)
at org.elasticsearch.index.translog.Translog.(Translog.java:179)
at org.elasticsearch.index.engine.InternalEngine.openTranslog(InternalEngine.java:208)
at org.elasticsearch.index.engine.InternalEngine.(InternalEngine.java:151)