# UnavailableShardsException: at least one primary shard for the index \[.security-7\] is unavailable

**URL:** https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393
**Category:** Elasticsearch
**Created:** [November 17, 2023, 9:37am UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393 "2023-11-17T09:37:55Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![rahul\_sirugudi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahul_sirugudi/32/94019_2.png) [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)
#### Post date: [November 17, 2023, 9:37am UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393/1 "2023-11-17T09:37:55Z")

</div>

Hi,  
I have deleted all files manually from /var/lib/elasticsearch/nodes/0/indices/. Now when i restart Elasticsearch server.

```auto
[2023-11-17T09:20:22,899][INFO][o.e.x.s.a.RealmsAuthenticator] [ip] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2023-11-17T09:20:23,721][ERROR][o.e.x.s.a.e.ReservedRealm] [ip] failed to retrieve password hash for reserved user [elastic]
org.elasticsearch.action.UnavailableShardsException: at least one primary shard for the index [.security-7] is unavailable

```

I don't have any backups for the snapshot any way i can make this working?

---

<div class="post-metadata">

### Author: ![rahul\_sirugudi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rahul_sirugudi/32/94019_2.png) [@rahul\_sirugudi](https://discuss.elastic.co/u/rahul_sirugudi)
#### Post date: [November 17, 2023, 9:59am UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393/2 "2023-11-17T09:59:59Z")

</div>

I have followed few steps mentioned on this 2 links.

> [@Accidentally deleted .security index for x-pack](https://discuss.elastic.co/t/accidentally-deleted-security-index-for-x-pack/69844/2):
>
> nm, it seems to have come back

and here

```auto
https://discuss.elastic.co/t/elk-7-8-0-two-node-cluster-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/261031/3

```

but this 2 steps didn't helped me.

---

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [November 17, 2023, 3:41pm UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393/3 "2023-11-17T15:41:15Z")

</div>

> [@rahul\_sirugudi](#):
>
> I don't have any backups for the snapshot any way i can make this working?

Nope, sorry, it sounds like you've destroyed all your data. See [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#data-path) for more information:

> **WARNING** : Don’t modify anything within the data directory or run processes that might interfere with its contents...

---

<div class="post-metadata">

### Author: ![NIK2501nc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik2501nc/32/126630_2.png) [@NIK2501nc](https://discuss.elastic.co/u/NIK2501nc)
#### Post date: [November 20, 2023, 9:03am UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393/4 "2023-11-20T09:03:48Z")

</div>

Hi @rahul_sirugudi

It seems your nodes are unable to store the system index (.security index). Try to change your master node role as data\_content by adding data\_content role in node\_roles of your nodes YML file.

Refer the below post for more detail-

> [@Failed to retrieve password hash for reserved user \[elastic\] org.elasticsearch.action.UnavailableShardsException: at least one primary shard for the index \[.security-7\] is unavailable](https://discuss.elastic.co/t/failed-to-retrieve-password-hash-for-reserved-user-elastic-org-elasticsearch-action-unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347499/1):
>
> failed to retrieve password hash for reserved user [elastic] org.elasticsearch.action.UnavailableShardsException: at least one primary shard for the index [.security-7] is unavailable You'll get this issue when in your cluster, the nodes are unable to contain the system index ( like ".index, .security, .kibana etc." ) for resolving this issue- Add the node role as "data\_content" the YML setting will be- node roles: [data\_content, other node roles whichever you want separated by comma] For D…

After changing the role UUID of the node will change so in such case for existing cluster you need to kill all elastic node process then change the node role and restart your other node which results your cluster up and after that you'll not face this issue.

incase of any doubt share your YML settings or if it resolves let us know for the same.

Good luck!

---

<div class="post-metadata">

### Author: ![NIK2501nc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik2501nc/32/126630_2.png) [@NIK2501nc](https://discuss.elastic.co/u/NIK2501nc)
#### Post date: [November 20, 2023, 10:28am UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393/5 "2023-11-20T10:28:44Z")

</div>

Hi @DavidTurner ,

We need to do some addition in our Elastic documentation so that it'll help to increase the deep understanding for developers and users about **Elastic System index**. I can see we don't have any significant explanation of System Index (security index) which is crucial Onwards Elastic Version 8 in cluster setup with dedicated master node.

There are lot of exceptions and errors available related to this concept directly or indirectly. Kindly ask the Developers & Documentation team to make the changes.  
Incase if it's already available kindly share the link for reference.

Thanks!

---

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [November 20, 2023, 11:03am UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393/6 "2023-11-20T11:03:48Z")

</div>

Although the index reporting the problem here happens to be a system index, this fact is not in any way relevant. You deleted all the data in the data path.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 18, 2023, 11:04am UTC](https://discuss.elastic.co/t/unavailableshardsexception-at-least-one-primary-shard-for-the-index-security-7-is-unavailable/347393/7 "2023-12-18T11:04:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
