# org.elasticsearch.cluster.block.ClusterBlockException: blocked by: \[SERVICE\_UNAVAILABLE/1/state not recovered / initialized\]
**URL:** https://discuss.elastic.co/t/org-elasticsearch-cluster-block-clusterblockexception-blocked-by-service-unavailable-1-state-not-recovered-initialized/309992
**Category:** Elasticsearch
**Created:** [July 19, 2022, 8:17am UTC](https://discuss.elastic.co/t/org-elasticsearch-cluster-block-clusterblockexception-blocked-by-service-unavailable-1-state-not-recovered-initialized/309992 "2022-07-19T08:17:47Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@RRE](https://discuss.elastic.co/u/RRE)
#### Post date: [July 19, 2022, 8:17am UTC](https://discuss.elastic.co/t/org-elasticsearch-cluster-block-clusterblockexception-blocked-by-service-unavailable-1-state-not-recovered-initialized/309992/1 "2022-07-19T08:17:47Z")
Hi, after a server reboot when I try to start elasticsearch I get the following error:
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE\_UNAVAILABLE/1/state not recovered / initialized]
this is followed by a lot of additional errors:
```auto
[2022-07-19T09:14:56,304][ERROR][o.e.x.m.c.m.JobStatsCollector] [am-elastic-acc-n1] collector [job_stats] failed to collect data
[2022-07-19T09:14:58,478][ERROR][o.e.x.m.c.c.ClusterStatsCollector] [am-elastic-acc-n1] collector [cluster_stats] failed to collect data
[2022-07-19T09:15:56,862][ERROR][o.e.x.w.i.s.ExecutableSearchInput] [am-elastic-acc-n1] failed to execute [search] input for watch [GYOADuWdQzGTxrQOIxxDLQ_logstash_version_mismatch], reason [all shards failed]
[2022-07-19T09:15:57,277][ERROR][o.e.x.w.i.s.ExecutableSearchInput] [am-elastic-acc-n1] failed to execute [search] input for watch [GYOADuWdQzGTxrQOIxxDLQ_xpack_license_expiration], reason [all shards failed]
[2022-07-19T09:15:57,280][ERROR][o.e.x.w.i.s.ExecutableSearchInput] [am-elastic-acc-n1] failed to execute [search] input for watch [GYOADuWdQzGTxrQOIxxDLQ_elasticsearch_version_mismatch], reason [all shards failed]
[2022-07-19T09:15:57,281][ERROR][o.e.x.w.i.s.ExecutableSearchInput] [am-elastic-acc-n1] failed to execute [search] input for watch [GYOADuWdQzGTxrQOIxxDLQ_elasticsearch_cluster_status], reason [all shards failed]
[2022-07-19T09:15:57,285][ERROR][o.e.x.w.i.s.ExecutableSearchInput] [am-elastic-acc-n1] failed to execute [search] input for watch [GYOADuWdQzGTxrQOIxxDLQ_elasticsearch_nodes], reason [all shards failed]
[2022-07-19T09:15:57,977][ERROR][o.e.x.w.i.s.ExecutableSearchInput] [am-elastic-acc-n1] failed to execute [search] input for watch [GYOADuWdQzGTxrQOIxxDLQ_kibana_version_mismatch], reason [all shards failed]
```
In the end if I keep elasticsearch running all my data is deleted
My environment has been running fine for months, I am running version 7.16.2
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 19, 2022, 10:06pm UTC](https://discuss.elastic.co/t/org-elasticsearch-cluster-block-clusterblockexception-blocked-by-service-unavailable-1-state-not-recovered-initialized/309992/2 "2022-07-19T22:06:43Z")
Can you share more of your Elasticsearch log?
---
### Author:  [@RRE](https://discuss.elastic.co/u/RRE)
#### Post date: [July 20, 2022, 7:01am UTC](https://discuss.elastic.co/t/org-elasticsearch-cluster-block-clusterblockexception-blocked-by-service-unavailable-1-state-not-recovered-initialized/309992/3 "2022-07-20T07:01:03Z")
Below some more logging
```auto
[2022-07-19T09:14:45,780][INFO][o.e.n.Node] [am-elastic-acc-n1] started
[2022-07-19T09:14:46,093][INFO][o.e.c.s.ClusterSettings] [am-elastic-acc-n1] updating [xpack.monitoring.collection.enabled] from [false] to [true]
[2022-07-19T09:14:46,424][WARN][o.e.x.s.a.ApiKeyAuthenticator] [am-elastic-acc-n1] Authentication using apikey failed - apikey authentication for id LqEtmYABfN3Wvu8XkwEC encountered a failure
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:179) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.checkGlobalBlock(TransportSingleShardAction.java:112) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:146) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:130) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:98) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:51) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:179) [elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) [elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) [elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:190) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.action.ActionListener$DelegatingFailureActionListener.onResponse(ActionListener.java:219) [elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:577) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.SearchRequestInterceptor.intercept(SearchRequestInterceptor.java:26) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.ShardSearchRequestInterceptor.intercept(ShardSearchRequestInterceptor.java:26) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.DlsFlsLicenseRequestInterceptor.intercept(DlsFlsLicenseRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.BulkShardRequestInterceptor.intercept(BulkShardRequestInterceptor.java:87) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.IndicesAliasesRequestInterceptor.intercept(IndicesAliasesRequestInterceptor.java:128) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]
at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]
```
and here some more
```auto
[2022-07-19T09:14:46,764][INFO][o.e.l.LicenseService] [am-elastic-acc-n1] license [2cec4340-38dc-482c-9875-efa058f0d40f] mode [platinum] - valid
[2022-07-19T09:14:46,764][INFO][o.e.x.s.a.Realms] [am-elastic-acc-n1] license mode is [platinum], currently licensed security realms are [reserved/reserved,native/native1,active_directory/apg_ad]
[2022-07-19T09:14:46,765][INFO][o.e.x.s.s.SecurityStatusChangeListener] [am-elastic-acc-n1] Active license is now [PLATINUM]; Security is enabled
[2022-07-19T09:14:46,769][INFO][o.e.g.GatewayService] [am-elastic-acc-n1] recovered [142] indices into cluster_state
[2022-07-19T09:14:47,206][INFO][o.e.x.i.IndexLifecycleTransition] [am-elastic-acc-n1] moving index [.watcher-history-13-2022.07.09] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"delete","action":"delete","name":"wait-for-shard-history-leases"}] in policy [watch-history-ilm-policy]
[2022-07-19T09:14:47,208][INFO][o.e.x.i.IndexLifecycleTransition] [am-elastic-acc-n1] moving index [.watcher-history-13-2022.07.04] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"delete","action":"delete","name":"wait-for-shard-history-leases"}] in policy [watch-history-ilm-policy]
[2022-07-19T09:14:47,209][INFO][o.e.x.i.IndexLifecycleTransition] [am-elastic-acc-n1] moving index [.watcher-history-13-2022.07.06] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"delete","action":"delete","name":"wait-for-shard-history-leases"}] in policy [watch-history-ilm-policy]
[2022-07-19T09:14:47,210][INFO][o.e.x.i.IndexLifecycleTransition] [am-elastic-acc-n1] moving index [.watcher-history-13-2022.07.03] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"delete","action":"delete","name":"wait-for-shard-history-leases"}] in policy [watch-history-ilm-policy]
[2022-07-19T09:14:47,213][INFO][o.e.x.i.IndexLifecycleTransition] [am-elastic-acc-n1] moving index [.watcher-history-13-2022.07.07] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"delete","action":"delete","name":"wait-for-shard-history-leases"}] in policy [watch-history-ilm-policy]
[2022-07-19T09:14:47,213][INFO][o.e.x.i.IndexLifecycleTransition] [am-elastic-acc-n1] moving index [.watcher-history-13-2022.07.08] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"delete","action":"delete","name":"wait-for-shard-history-leases"}] in policy [watch-history-ilm-policy]
[2022-07-19T09:14:47,214][INFO][o.e.x.i.IndexLifecycleTransition] [am-elastic-acc-n1] moving index [.watcher-history-13-2022.07.05] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"delete","action":"delete","name":"wait-for-shard-history-leases"}] in policy [watch-history-ilm-policy]
[2022-07-19T09:14:55,617][INFO][o.e.x.w.WatcherService] [am-elastic-acc-n1] reloading watcher, reason [new local watcher shard allocation ids], cancelled [0] queued tasks
[2022-07-19T09:14:56,304][ERROR][o.e.x.m.c.m.JobStatsCollector] [am-elastic-acc-n1] collector [job_stats] failed to collect data
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:725) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:412) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:757) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:509) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.performPhaseOnShard(AbstractSearchAsyncAction.java:320) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:256) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:466) ~[elasticsearch-7.16.2.jar:7.16.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:211) ~[elasticsearch-7.16.
```
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 20, 2022, 10:36pm UTC](https://discuss.elastic.co/t/org-elasticsearch-cluster-block-clusterblockexception-blocked-by-service-unavailable-1-state-not-recovered-initialized/309992/4 "2022-07-20T22:36:26Z")
It looks like you have a platinum license? If so definitely raise a request with the Support team.
It does look like Elasticsearch is trying to recover, what is the status of it?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 17, 2022, 10:36pm UTC](https://discuss.elastic.co/t/org-elasticsearch-cluster-block-clusterblockexception-blocked-by-service-unavailable-1-state-not-recovered-initialized/309992/5 "2022-08-17T22:36:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.