Hello,
We are on a two node ELK cluster and noticing the below error suddennly. As the error says .security index seems to be corrupted. Is there anyway to restore the old state?
Due to this kibana is not loading, none of the user authentication works as well.
ELK : Version 7.8.0
[2021-01-13T17:05:27,359][INFO ][o.e.x.s.a.AuthenticationService] [elekpelk01] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2021-01-13T17:05:29,290][ERROR][o.e.x.s.a.e.ReservedRealm] [elekpelk01] failed to retrieve password hash for reserved user [elastic]
org.elasticsearch.action.UnavailableShardsException: at least one primary shard for the index [.security-7] is unavailable
at org.elasticsearch.xpack.security.support.SecurityIndexManager.getUnavailableReason(SecurityIndexManager.java:181) ~[x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.authc.esnative.NativeUsersStore.getReservedUserInfo(NativeUsersStore.java:525) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.getUserInfo(ReservedRealm.java:224) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.doAuthenticate(ReservedRealm.java:99) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticateWithCache(CachingUsernamePasswordRealm.java:167) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:104) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$consumeToken$15(AuthenticationService.java:449) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:102) [x-pack-core-7.8.0.jar:7.8.0]