# Cluster Health status is Red **URL:** \ **Category:** Elasticsearch\ **Created:** [September 15, 2021, 7:10pm UTC](https://discuss.elastic.co/t/cluster-health-status-is-red/284320 "2021-09-15T19:10:23Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![rachelyang](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@rachelyang](https://discuss.elastic.co/u/rachelyang)\ **Post date:** [September 15, 2021, 7:10pm UTC](https://discuss.elastic.co/t/cluster-health-status-is-red/284320/1 "2021-09-15T19:10:23Z")
I am establishing a cluster: 1 master node + 1 backup node. The cluster health status is red. curl -X GET "localhost:9200/\_cluster/health?pretty" { "cluster\_name" : "Elasticsearch", "status" : "red", "timed\_out" : false, "number\_of\_nodes" : 2, "number\_of\_data\_nodes" : 2, "active\_primary\_shards" : 22, "active\_shards" : 44, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 4, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 0, "number\_of\_in\_flight\_fetch" : 0, "task\_max\_waiting\_in\_queue\_millis" : 0, "active\_shards\_percent\_as\_number" : 91.66666666666666 } I have one index which is red. curl -XGET localhost:9200/\_cat/indices | grep -i red % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 2438 100 2438 0 0 238k 0 --:--:-- --:--:-- --:--:-- 238k red open .geoip\_databases zMiIvdOiSk-ffIbsN1aTew 1 1 But this index is system purpose. I cannot delete it. curl -X DELETE "localhost:9200/.geoip\_databases/" {"error":{"root\_cause":[{"type":"illegal\_argument\_exception","reason":"Indices [.geoip\_databases] use and access is reserved for system operations"}],"type":"illegal\_argument\_exception","reason":"Indices [.geoip\_databases] use and access is reserved for system operations"},"status":400} How can I fix the red status issue? Thanks. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 15, 2021, 10:23pm UTC](https://discuss.elastic.co/t/cluster-health-status-is-red/284320/2 "2021-09-15T22:23:28Z")
Welcome to our community! 😃 What version are you on? What do your Elasticsearch logs show? ---
**Author:** ![rachelyang](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@rachelyang](https://discuss.elastic.co/u/rachelyang)\ **Post date:** [September 16, 2021, 1:19pm UTC](https://discuss.elastic.co/t/cluster-health-status-is-red/284320/3 "2021-09-16T13:19:23Z")
My Elasticsearch version is 7.14.1. This is the health check result: curl -X GET "localhost:9200/\_cluster/health?pretty" { "cluster\_name" : "Elasticsearch", "status" : "red", "timed\_out" : false, "number\_of\_nodes" : 2, "number\_of\_data\_nodes" : 2, "active\_primary\_shards" : 30, "active\_shards" : 60, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 4, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 0, "number\_of\_in\_flight\_fetch" : 0, "task\_max\_waiting\_in\_queue\_millis" : 0, "active\_shards\_percent\_as\_number" : 93.75 } This is my unassigned shards info: curl -X GET localhost:9200/\_cat/shards?h=index,shard,prirep,state,unassigned.reason | grep UNASSIGNED % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3580 100 3580 0 0 124k 0 --:--:-- --:--:-- --:--:-- 124k .ds-ilm-history-5-2021.08.26-000002 0 p UNASSIGNED CLUSTER\_RECOVERED .ds-ilm-history-5-2021.08.26-000002 0 r UNASSIGNED REPLICA\_ADDED .geoip\_databases 0 r UNASSIGNED REPLICA\_ADDED .geoip\_databases 0 p UNASSIGNED CLUSTER\_RECOVERED I cannot do anything on these shards. I have a warning a an error in the Elasticsearch.log file. [WARN][r.suppressed] [node1] path: /.kibana\_7.14.1/\_doc/space%3Adefault, params: {index=.kibana\_7.14.1, id=space:default} org.elasticsearch.action.NoShardAvailableActionException: No shard available for [get [.kibana\_7.14.1][\_doc][space:default]: routing [null]] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.perform(TransportSingleShardAction.java:208) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.start(TransportSingleShardAction.java:185) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:92) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:51) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:173) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:42) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:171) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:122) [x-pack-security-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:171) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:149) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:77) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:86) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:66) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:402) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.client.support.AbstractClient.get(AbstractClient.java:504) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.rest.action.document.RestGetAction.lambda$prepareRequest$0(RestGetAction.java:86) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:106) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:98) [x-pack-security-7.14.1.jar:7.14.1] at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:274) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:356) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:195) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:471) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:536) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:398) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:31) [transport-netty4-client-7.14.1.jar:7.14.1] at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:17) [transport-netty4-client-7.14.1.jar:7.14.1] at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:47) [transport-netty4-client-7.14.1.jar:7.14.1] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final] at java.lang.Thread.run(Thread.java:831) [?:?] ---
**Author:** ![rachelyang](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@rachelyang](https://discuss.elastic.co/u/rachelyang)\ **Post date:** [September 16, 2021, 1:19pm UTC](https://discuss.elastic.co/t/cluster-health-status-is-red/284320/4 "2021-09-16T13:19:40Z")
[ERROR][o.e.i.g.GeoIpDownloader] [node1] error updating geoip database [GeoLite2-ASN.mmdb] org.elasticsearch.action.UnavailableShardsException: [.geoip\_databases][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.geoip\_databases][0]] containing [index {[.geoip\_databases][\_doc][GeoLite2-ASN.mmdb\_20\_1631797749539], source[n/a, actual length: [1mb], max length: 2kb]}]] at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase.retryBecauseUnavailable(TransportReplicationAction.java:885) ~[elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase.doRun(TransportReplicationAction.java:735) ~[elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase$2.onTimeout(TransportReplicationAction.java:845) ~[elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.cluster.ClusterStateObserver$ContextPreservingListener.onTimeout(ClusterStateObserver.java:324) ~[elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:241) ~[elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.cluster.service.ClusterApplierService$NotifyTimeout.run(ClusterApplierService.java:590) ~[elasticsearch-7.14.1.jar:7.14.1] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673) ~[elasticsearch-7.14.1.jar:7.14.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.lang.Thread.run(Thread.java:831) [?:?] ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 14, 2021, 1:19pm UTC](https://discuss.elastic.co/t/cluster-health-status-is-red/284320/5 "2021-10-14T13:19:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.