# Shards got deleted after adding/removing index.number\_of\_replicas : 2

**URL:** https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875
**Category:** Elasticsearch
**Created:** [March 31, 2017, 9:52pm UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875 "2017-03-31T21:52:44Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Jayanna\_Hallur](https://avatars.discourse-cdn.com/v4/letter/j/a4c791/32.png) [@Jayanna\_Hallur](https://discuss.elastic.co/u/Jayanna_Hallur)
#### Post date: [March 31, 2017, 9:52pm UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875/1 "2017-03-31T21:52:44Z")

</div>

Hi All

I had ES cluster with 3 (data+master) & 1 co-ordinator node. I wanted set the default replication to 2, hence i following the below steps.

The ES cluster was fine and there were more than 100 shards and working well since 3-4 weeks.

1. Logged into all 4 ES nodes & killed the elastic daemon
2. Added the below lines to elasticsearch.yml in all 4 nodes  
index.number\_of\_replicas : 2
3. Restarted the ES node..
4. ES nodes didnt start..looking into the log and found information as specified below
5. Removed the index.number\_of\_replicas : 2 from yml file
6. restarted all the ES nodes..
7. Cluster is up and running..

The min master settings:  
discovery.zen.minimum\_master\_nodes: 3

My Question here is :  
when I issue /\_cat/indices?v , i see only .kibana index and the 100+ olds ones got deleted. Why the shards got deleted?

Error in log file when ES started with index.number\_of\_replicas : 2 in config file..

[2017-03-31T14:38:54,798][WARN][o.e.c.s.SettingsModule] [dev\_es\_node\_3a]

* * *

Found index level settings on node level configuration.

Since elasticsearch 5.x index level settings can NOT be set on the nodes  
configuration like the elasticsearch.yaml, in system properties or command line  
[arguments.In](http://arguments.In) order to upgrade all indices the settings must be updated via the  
/${index}/\_settings API. Unless all settings are dynamic all indices must be closed  
in order to apply the upgradeIndices created in the future should use index templates  
to set default values.

Please ensure all required values are updated on all indices by executing:

curl -XPUT '[http://localhost:9200/\_all/\_settings?preserve\_existing=true](http://localhost:9200/_all/_settings?preserve_existing=true)' -d '{  
"index.number\_of\_replicas" : "2"  
}'

* * *

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [March 31, 2017, 10:24pm UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875/2 "2017-03-31T22:24:49Z")

</div>

Is there anything else that you changed in the config?

---

<div class="post-metadata">

### Author: ![Jayanna\_Hallur](https://avatars.discourse-cdn.com/v4/letter/j/a4c791/32.png) [@Jayanna\_Hallur](https://discuss.elastic.co/u/Jayanna_Hallur)
#### Post date: [March 31, 2017, 10:26pm UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875/3 "2017-03-31T22:26:35Z")

</div>

No.. only 1 line ..  
index.number\_of\_replicas : 2

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [March 31, 2017, 10:45pm UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875/4 "2017-03-31T22:45:53Z")

</div>

That's highly unusual.

What's `_cat/indices` show?  
Can you see directories for the original indices on the filesystem?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 1, 2017, 7:40am UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875/5 "2017-04-01T07:40:11Z")

</div>

> [@Jayanna\_Hallur](#):
>
> The min master settings:discovery.zen.minimum\_master\_nodes: 3

If you have 3 master eligible nodes, this should be set to 2. I you have it set to 3 the cluster will not be able to elect a master once one of the master eligible nodes go offline, which will cause problems.

---

<div class="post-metadata">

### Author: ![Jayanna\_Hallur](https://avatars.discourse-cdn.com/v4/letter/j/a4c791/32.png) [@Jayanna\_Hallur](https://discuss.elastic.co/u/Jayanna_Hallur)
#### Post date: [April 3, 2017, 5:28pm UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875/6 "2017-04-03T17:28:33Z")

</div>

But with the same config, there were so many index & shards were created... if one goes down, as per the documentation the write will fail.. but why the shards got deleted when the all masters are back?

There is no brain split scenario here...

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 1, 2017, 5:28pm UTC](https://discuss.elastic.co/t/shards-got-deleted-after-adding-removing-index-number-of-replicas-2/80875/7 "2017-05-01T17:28:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
