# Kibana server not starting up **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-security\ **Created:** [December 8, 2020, 7:09am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944 "2020-12-08T07:09:52Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 8, 2020, 7:09am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/1 "2020-12-08T07:09:52Z")
iam seeing this error in the logs.It was working fine till 1 day back.Whta could be the issue here. i restarted the kibana.Still the same issue.ECerything was properly setup. {"type":"log","@timestamp":"2020-12-08T06:53:00Z","tags":["reporting","warning"],"pid":1,"message":"Reporting plugin self-check failed. Please check the Kibana Reporting settings. [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } } :: {"path":"/\_cluster/settings","query":{"include\_defaults":true},"statusCode":401,"response":"{\"error\":{\"root\_cause\":[{\"type\":\"security\_exception\",\"reason\":\"failed to authenticate user [elastic]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}}],\"type\":\"security\_exception\",\"reason\":\"failed to authenticate user [elastic]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}},\"status\":401}","wwwAuthenticateDirective":"Basic realm=\"security\" charset=\"UTF-8\""}"} {"type":"log","@timestamp":"2020-12-08T06:53:00Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:00Z","tags":["warning","maps"],"pid":1,"message":"Error scheduling telemetry task, received NotInitialized: Tasks cannot be scheduled until after task manager is initialized!"} {"type":"log","@timestamp":"2020-12-08T06:53:00Z","tags":["warning","telemetry"],"pid":1,"message":"Error scheduling task, received NotInitialized: Tasks cannot be scheduled until after task manager is initialized!"} {"type":"log","@timestamp":"2020-12-08T06:53:03Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:06Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:09Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:12Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:15Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:18Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:21Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:24Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:27Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:28Z","tags":["license","warning","xpack"],"pid":1,"message":"License information from the X-Pack plugin could not be obtained from Elasticsearch for the [data] cluster. [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } } :: {"path":"/\_xpack","statusCode":401,"response":"{\"error\":{\"root\_cause\":[{\"type\":\"security\_exception\",\"reason\":\"failed to authenticate user [elastic]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}}],\"type\":\"security\_exception\",\"reason\":\"failed to authenticate user [elastic]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}},\"status\":401}","wwwAuthenticateDirective":"Basic realm=\"security\" charset=\"UTF-8\""}"} {"type":"log","@timestamp":"2020-12-08T06:53:30Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:33Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:36Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} {"type":"log","@timestamp":"2020-12-08T06:53:39Z","tags":["warning","task\_manager"],"pid":1,"message":"PollError [security\_exception] failed to authenticate user [elastic], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"} ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 8, 2020, 7:12am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/2 "2020-12-08T07:12:25Z")
> [@sudhakar\_u](#): > > failed to authenticate user [elastic] Has something changed with the default user? Can you query the Elasticsearch APIs using the details you have set? ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 8, 2020, 8:02am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/3 "2020-12-08T08:02:25Z")
This is what i could see in my elastic client node.Nothing has been changed. {"type": "server", "timestamp": "2020-12-08T07:58:35,729+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "Gs7d-Bl0Qsi1jjqz0uv9HA", "node.id": "KWjHxe7\_RH-6VBs5tGSm\_w", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } {"type": "server", "timestamp": "2020-12-08T07:58:37,335+0000", "level": "ERROR", "component": "o.e.x.s.a.e.ReservedRealm", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "Gs7d-Bl0Qsi1jjqz0uv9HA", "node.id": "KWjHxe7\_RH-6VBs5tGSm\_w", "message": "failed to retrieve password hash for reserved user [elastic]" , "stacktrace": ["org.elasticsearch.action.UnavailableShardsException: at least one primary shard for the index [.security-7] is unavailable", "at org.elasticsearch.xpack.security.support.SecurityIndexManager.getUnavailableReason(SecurityIndexManager.java:181) ~[x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.esnative.NativeUsersStore.getReservedUserInfo(NativeUsersStore.java:525) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.getUserInfo(ReservedRealm.java:212) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.doAuthenticate(ReservedRealm.java:93) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticateWithCache(CachingUsernamePasswordRealm.java:166) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:103) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$consumeToken$15(AuthenticationService.java:364) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:102) [x-pack-core-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:407) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$11(AuthenticationService.java:335) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:345) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$checkForApiKey$3(AuthenticationService.java:288) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.ApiKeyService.authenticateWithApiKeyIfPresent(ApiKeyService.java:360) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.checkForApiKey(AuthenticationService.java:269) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$0(AuthenticationService.java:252) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.TokenService.getAndValidateToken(TokenService.java:390) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:248) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$6(AuthenticationService.java:306) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:317) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:244) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:196) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:122) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:55) [x-pack-security-7.3.0.jar:7.3.0]", "at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:240) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:344) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:174) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:320) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:370) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:299) [elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:66) [transport-netty4-client-7.3.0.jar:7.3.0]", "at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:31) [transport-netty4-client-7.3.0.jar:7.3.0]", "at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:58) [transport-netty4-client-7.3.0.jar:7.3.0]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) [netty-codec-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287) [netty-handler-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:682) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:582) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:536) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) [netty-transport-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906) [netty-common-4.1.36.Final.jar:4.1.36.Final]", "at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.36.Final.jar:4.1.36.Final]", "at java.lang.Thread.run(Thread.java:835) [?:?]"] } {"type": "server", "timestamp": "2020-12-08T07:58:37,336+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "Gs7d-Bl0Qsi1jjqz0uv9HA", "node.id": "KWjHxe7\_RH-6VBs5tGSm\_w", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 8, 2020, 9:38pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/4 "2020-12-08T21:38:34Z")
You still have auth issues, so you need to check you can connect to Elasticsearch using the API and the details you have. ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 8, 2020, 10:05pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/5 "2020-12-08T22:05:08Z")
> [@sudhakar\_u](#): > > Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } i tried with curl..i keep getting this {"error":{"root\_cause":[{"type":"security\_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}}],"type":"security\_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401} Looks strange to me.Nothing is chnaged suddenly elastic user is failing. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 8, 2020, 10:06pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/6 "2020-12-08T22:06:15Z")
Sounds like something has changed then. Do you have other users you can try? ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 8, 2020, 10:27pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/7 "2020-12-08T22:27:29Z")
> [@sudhakar\_u](#): > > "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } i tried with another usr also with no luck curl -u sudhakar '[http://elasticsearch-client.logging.svc.cluster.local:9200/\_xpack/security/\_authenticate?pretty](http://elasticsearch-client.logging.svc.cluster.local:9200/_xpack/security/_authenticate?pretty)' Enter host password for user 'sudhakar': { "error" : { "root\_cause" : [ { "type" : "security\_exception", "reason" : "unable to authenticate user [sudhakar] for REST request [/\_xpack/security/\_authenticate?pretty]", "header" : { "WWW-Authenticate" : "Basic realm="security" charset="UTF-8"" } } ], "type" : "security\_exception", "reason" : "unable to authenticate user [sudhakar] for REST request [/\_xpack/security/\_authenticate?pretty]", "header" : { "WWW-Authenticate" : "Basic realm="security" charset="UTF-8"" } }, "status" : 401 THis is Elasticsearch 7.3 and iam running a docker image. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 8, 2020, 10:30pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/8 "2020-12-08T22:30:09Z")
> [@sudhakar\_u](#): > > at least one primary shard for the index [.security-7] is unavailable Ok this is a bit of a problem. Did something happen to your cluster recently? ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 8, 2020, 10:39pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/9 "2020-12-08T22:39:49Z")
elasticsearch-client-56db759bc-ds4c8 1/1 Running 0 15d elasticsearch-data-0 1/1 Running 0 3d19h elasticsearch-master-645ccc5b4c-nl2fp 1/1 Running 0 15d not that iam aware of iam seeing this in my master node {"type": "server", "timestamp": "2020-12-08T22:37:31,434+0000", "level": "ERROR", "component": "o.e.x.m.c.c.ClusterStatsCollector", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master", "cluster.uuid": "Gs7d-Bl0Qsi1jjqz0uv9HA", "node.id": "Yn6TOUnHT6WlKbBW8qhnFw", "message": "collector [cluster\_stats] failed to collect data" , "stacktrace": ["org.elasticsearch.action.UnavailableShardsException: at least one primary shard for the index [.security-7] is unavailable", "at org.elasticsearch.xpack.security.support.SecurityIndexManager.getUnavailableReason(SecurityIndexManager.java:181) ~[?:?]", "at org.elasticsearch.xpack.security.authc.esnative.NativeUsersStore.getUserCount(NativeUsersStore.java:170) ~[?:?]", "at org.elasticsearch.xpack.security.authc.esnative.NativeRealm.lambda$usageStats$1(NativeRealm.java:53) ~[?:?]", "at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.lambda$usageStats$5(CachingUsernamePasswordRealm.java:203) ~[?:?]", "at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.3.0.jar:7.3.0]", "at org.elasticsearch.xpack.core.security.authc.Realm.usageStats(Realm.java:137) ~[?:?]", "at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.usageStats(CachingUsernamePasswordRealm.java:201) ~[?:?]", "at org.elasticsearch.xpack.security.authc.esnative.NativeRealm.usageStats(NativeRealm.java:52) ~[?:?]", "at org.elasticsearch.xpack.security.authc.Realms.usageStats(Realms.java:268) ~[?:?]", "at org.elasticsearch.xpack.security.SecurityFeatureSet.usage(SecurityFeatureSet.java:143) ~[?:?]" ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 8, 2020, 10:40pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/10 "2020-12-08T22:40:32Z")
I am not sure how to recover from this sorry. Hopefully one of the security engineers can take a look. ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 8, 2020, 10:41pm UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/11 "2020-12-08T22:41:29Z")
If i restart my nodes..will i loose data? ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [December 9, 2020, 1:05am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/12 "2020-12-09T01:05:54Z")
At this point there's a chance you've already lost data. The only thing we know is that one of your indices (`.security-7`) is missing its primary shard. Perhaps other indices are missing as well, we won't know until you can get the cluster into a state where you can access the management APIs and see what's going on. If you have a paid support contract, you should log a case, so that you can get expert help working out what's happened and how to resolve it. If not... 1. You need to work out whether you have the [file realm](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/file-realm.html#file-realm-configuration) enabled. By default, you will, but if you have made changes to your `elasticsearch.yml` and have changed the realm configuration, then you might not. - If it is not enabled then you will need to enable it on 1 node and restart that node. 2. Use the [`elasticsearch-users`](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/users-command.html) utility to add a new superuser to your cluster ```auto elasticsearch-users useradd admin -p password -r superuser ``` 3. Use that new `admin` user to check the [cluster health](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/cluster-health.html) and the [status of your shards](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/cat-shards.html). ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 9, 2020, 3:41am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/13 "2020-12-09T03:41:45Z")
i have restarted my nodes **my elastic master has this status** {"type": "server", "timestamp": "2020-12-08T22:56:38,781+0000", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "dg6m\_iePT1CW2ImOFE6drA", "message": "Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.monitoring-es-7-2020.12.08][0]] ...])." } My elastic lcient has this error {"type": "server", "timestamp": "2020-12-09T03:36:39,597+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "r-ZQVtM9Q0OATheKUumOag", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } {"type": "server", "timestamp": "2020-12-09T03:36:40,897+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "r-ZQVtM9Q0OATheKUumOag", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } {"type": "server", "timestamp": "2020-12-09T03:36:42,178+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "r-ZQVtM9Q0OATheKUumOag", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } {"type": "server", "timestamp": "2020-12-09T03:36:42,178+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "r-ZQVtM9Q0OATheKUumOag", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } {"type": "server", "timestamp": "2020-12-09T03:36:42,269+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "r-ZQVtM9Q0OATheKUumOag", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } {"type": "server", "timestamp": "2020-12-09T03:36:42,269+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "r-ZQVtM9Q0OATheKUumOag", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } i have the basic licesnse.i have setup in other test environments and everything is working fine.Only this environment its having this strange issue. I have restarted kibana {"type":"response","@timestamp":"2020-12-09T02:37:29Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"[example.com](http://example.com)","user-agent":"Go-http-client/1.1","accept-encoding":"gzip"},"remoteAddress":"10.240.0.8","userAgent":"10.240.0.8"},"res":{"statusCode":404,"responseTime":19,"contentLength":9},"message":"GET / 404 19ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:29Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/TP/public/index.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.6","userAgent":"10.240.0.6"},"res":{"statusCode":404,"responseTime":100,"contentLength":9},"message":"GET /TP/public/index.php 404 100ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:30Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/TP/index.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.5","userAgent":"10.240.0.5"},"res":{"statusCode":404,"responseTime":19,"contentLength":9},"message":"GET /TP/index.php 404 19ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:30Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/thinkphp/html/public/index.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.8","userAgent":"10.240.0.8"},"res":{"statusCode":404,"responseTime":19,"contentLength":9},"message":"GET /thinkphp/html/public/index.php 404 19ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:31Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/html/public/index.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.8","userAgent":"10.240.0.8"},"res":{"statusCode":404,"responseTime":17,"contentLength":9},"message":"GET /html/public/index.php 404 17ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:31Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/public/index.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.6","userAgent":"10.240.0.6"},"res":{"statusCode":404,"responseTime":99,"contentLength":9},"message":"GET /public/index.php 404 99ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:32Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/TP/html/public/index.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.5","userAgent":"10.240.0.5"},"res":{"statusCode":404,"responseTime":18,"contentLength":9},"message":"GET /TP/html/public/index.php 404 18ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:33Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/elrekt.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.6","userAgent":"10.240.0.6"},"res":{"statusCode":404,"responseTime":18,"contentLength":9},"message":"GET /elrekt.php 404 18ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:33Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/index.php","method":"get","headers":{"host":"20.186.48.230","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.240.0.5","userAgent":"10.240.0.5"},"res":{"statusCode":404,"responseTime":100,"contentLength":9},"message":"GET /index.php 404 100ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T02:44:34Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"20.186.48.230:80","user-agent":"Mozilla/5.0 (Windows; U; Windows NT 6.0;en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6)","connection":"close","accept-encoding":"gzip"},"remoteAddress":"10.244.5.1","userAgent":"10.244.5.1"},"res":{"statusCode":404,"responseTime":20,"contentLength":9},"message":"GET / 404 20ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T03:13:13Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"20.186.48.230:80","user-agent":"Mozilla/5.0 (compatible; Nimbostratus-Bot/v1.3.2; [http://cloudsystemnetworks.com](http://cloudsystemnetworks.com))"},"remoteAddress":"10.240.0.6","userAgent":"10.240.0.6"},"res":{"statusCode":404,"responseTime":102,"contentLength":9},"message":"GET / 404 102ms - 9.0B"} {"type":"response","@timestamp":"2020-12-09T03:22:53Z","tags":,"pid":1,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"20.186.48.230:80","user-agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36","content-length":"0"},"remoteAddress":"10.240.0.5","userAgent":"10.240.0.5"},"res":{"statusCode":404,"responseTime":20,"contentLength":9},"message":"GET / 404 20ms - 9.0B"} iam kibana UI is not laoding properly i have reset new password also for elastic during restarting of my elasric cluster ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 9, 2020, 3:47am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/14 "2020-12-09T03:47:18Z")
> [@sudhakar\_u](#): > > i have reset new password also for elastic during restarting of my elasric cluster Did you test those new details to make sure they work? Did you update Kibana? ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 9, 2020, 3:54am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/15 "2020-12-09T03:54:24Z")
Yes updated kibana also. ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 9, 2020, 3:56am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/16 "2020-12-09T03:56:42Z")
My kibana UI is not loading Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'nonce-rKogLRTK6yL0L/+A'". Either the 'unsafe-inline' keyword, a hash ('sha256-SHHSeLc0bp6xt4BoVVyUy+3IbVqp3ujLaR+s+kSP5UI='), or a nonce ('nonce-...') is required to enable inline execution. bootstrap.js:10 ^ A single error about an inline script not firing due to content security policy is expected! index.light.css:1 Failed to load resource: the server responded with a status of 503 () index.light.css:1 Failed to load resource: the server responded with a status of 503 () bootstrap.js:32 Uncaught TypeError: Cannot read property 'dataset' of null at HTMLLinkElement.failure (bootstrap.js:32) index.light.css:1 Failed to load resource: the server responded with a status of 503 () index.light.css:1 Failed to load resourc and elastic lient has the error pasted in below. {"type": "server", "timestamp": "2020-12-09T03:36:42,178+0000", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-client", "cluster.uuid": "VPIMafK0Sm2-WBwsS4u\_iA", "node.id": "r-ZQVtM9Q0OATheKUumOag", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]" } ---
**Author:** ![sudhakar\_u](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@sudhakar\_u](https://discuss.elastic.co/u/sudhakar_u)\ **Post date:** [December 9, 2020, 4:42am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/17 "2020-12-09T04:42:37Z")
My cluster status is green. this is the o/p of the cluster health {"cluster\_name":"elasticsearch","status":"green","timed\_out":false,"number\_of\_nodes":3,"number\_of\_data\_nodes":1,"active\_primary\_shards":7,"active\_shards":7,"relocating\_shards":0,"initializing\_shards":0,"unassigned\_shards":0,"delayed\_unassigned\_shards":0,"number\_of\_pending\_tasks":0,"number\_of\_in\_flight\_fetch":0,"task\_max\_waiting\_in\_queue\_millis":0,"active\_shards\_percent\_as\_number":100.0} ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 6, 2021, 4:42am UTC](https://discuss.elastic.co/t/kibana-server-not-starting-up/257944/18 "2021-01-06T04:42:39Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.