# Replica shards are sometimes not created **URL:** \ **Category:** Elasticsearch\ **Created:** [February 11, 2016, 4:33am UTC](https://discuss.elastic.co/t/replica-shards-are-sometimes-not-created/41451 "2016-02-11T04:33:56Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![angelsmile](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@angelsmile](https://discuss.elastic.co/u/angelsmile)\ **Post date:** [February 11, 2016, 4:33am UTC](https://discuss.elastic.co/t/replica-shards-are-sometimes-not-created/41451/1 "2016-02-11T04:33:57Z")
We are running elasticsearch 1.7.1 with two nodes. Node 2 was added recently. Number of primary shards has been 5 from beginning and that of replica was set to 1 recently when the 2nd node was added. elasticsearch configuration is as follows: Node 1: index.number\_of\_shards: 5 index.number\_of\_replicas: 1 node.master: true node.data: true discovery.zen.minimum\_master\_nodes: 1 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["node1", "node2"] Node 2: index.number\_of\_shards: 5 index.number\_of\_replicas: 1 node.master: true node.data: true discovery.zen.minimum\_master\_nodes: 1 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["node1", "node2"] The cluster status is green: [root@node1 ~]# curl localhost:9200/\_cluster/health?pretty { "cluster\_name" : "cluster", "status" : "green", "timed\_out" : false, "number\_of\_nodes" : 2, "number\_of\_data\_nodes" : 2, "active\_primary\_shards" : 4804, "active\_shards" : 5074, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 0, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 0, "number\_of\_in\_flight\_fetch" : 0 } However, the replica shards are sometimes created and sometimes not. Can you think of any reason for this? The following shard status shows no replicas created for indices logstash-2016.02.07 and logstash-2016.02.10. [root@node1 ~]# curl localhost:9200/\_cat/shards | grep -v "p STARTED" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 371k 100 371k 0 0 1184k 0 --:--:-- --:--:-- --:--:-- 1183k logstash-2016.02.06 4 r STARTED 1126096 316.8mb 127.0.0.1 node2 logstash-2016.02.06 0 r STARTED 1126188 316.1mb 127.0.0.1 node2 logstash-2016.02.06 3 r STARTED 1125715 317.1mb 127.0.0.1 node2 logstash-2016.02.06 1 r STARTED 1126635 316.8mb 127.0.0.1 node2 logstash-2016.02.06 2 r STARTED 1125514 313.6mb 127.0.0.1 node2 logstash-2016.02.09 2 r STARTED 1293958 386.5mb 127.0.0.1 node2 logstash-2016.02.09 0 r STARTED 1294749 388.3mb 127.0.0.1 node2 logstash-2016.02.09 3 r STARTED 1295572 388.4mb 127.0.0.1 node2 logstash-2016.02.09 1 r STARTED 1293890 386.7mb 127.0.0.1 node2 logstash-2016.02.09 4 r STARTED 1293994 387.9mb 127.0.0.1 node2 logstash-2016.02.08 4 r STARTED 1241764 364.4mb 127.0.0.1 node2 logstash-2016.02.08 0 r STARTED 1242733 365.5mb 127.0.0.1 node2 logstash-2016.02.08 3 r STARTED 1241681 366.3mb 127.0.0.1 node2 logstash-2016.02.08 1 r STARTED 1242307 366.4mb 127.0.0.1 node2 logstash-2016.02.08 2 r STARTED 1241895 366.2mb 127.0.0.1 node2 logstash-2016.02.11 4 r STARTED 292339 93.5mb 127.0.0.1 node2 logstash-2016.02.11 0 r STARTED 292805 86.9mb 127.0.0.1 node2 logstash-2016.02.11 3 r STARTED 292760 107.2mb 192.168.x.x node1 logstash-2016.02.11 1 r STARTED 292733 87.3mb 192.168.x.x node1 logstash-2016.02.11 2 r STARTED 292596 94mb 127.0.0.1 node2 ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 11, 2016, 5:14am UTC](https://discuss.elastic.co/t/replica-shards-are-sometimes-not-created/41451/2 "2016-02-11T05:14:19Z")
A few things; 1. 5 primaries is too many for that data set size, I'd reduce it to 1. 2. 2 nodes is bad, you should have 3 to ensure you get a majority quorum. > [@angelsmile](#): > > However, the replica shards are sometimes created and sometimes not I can't see `logstash-2016.02.07` or `logstash-2016.02.10` at all, so it's hard to say. What do the settings for those two indices show? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 11, 2016, 5:15am UTC](https://discuss.elastic.co/t/replica-shards-are-sometimes-not-created/41451/3 "2016-02-11T05:15:38Z")
With two master eligible nodes the minimum\_master\_nodes parameter should be set to 2 in order to prevent split brains. Having 3 master eligible nodes (minimum\_master\_nodes still set to 2) as Mark suggests is much better in a consensus based system. ---
**Author:** ![angelsmile](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@angelsmile](https://discuss.elastic.co/u/angelsmile)\ **Post date:** [February 11, 2016, 5:26am UTC](https://discuss.elastic.co/t/replica-shards-are-sometimes-not-created/41451/4 "2016-02-11T05:26:38Z")
Hi Mark and Christian, Thank you for your quick response. Yes I understand having two master eligible nodes is bad. How about add a new node to one of the two physical servers the current two nodes are running on? Will that make it better to prevent split brains? I will change minimum\_master\_nodes to 2 but do you think that is related to the issue? I'll also reduce the number of primary shards. Regarding logstash-2016.02.07 and logstash-2016.02.10, that's because I filtered to display only replica shards and these indices don't have replicas. [root@kibana ~]# curl localhost:9200/\_cat/shards | grep logstash-2016.02.07 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 373k 100 373k 0 0 671k 0 --:--:-- --:--:-- --:--:-- 673k logstash-2016.02.07 2 p STARTED 1122289 324.9mb 127.0.0.1 node2 logstash-2016.02.07 0 p STARTED 1122940 325.9mb 127.0.0.1 node2 logstash-2016.02.07 3 p STARTED 1122723 324.9mb 192.168.x.x node1 logstash-2016.02.07 1 p STARTED 1122855 325.5mb 127.0.0.1 node2 logstash-2016.02.07 4 p STARTED 1123432 323.5mb 192.168.x.x node1 ---
**Author:** ![angelsmile](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@angelsmile](https://discuss.elastic.co/u/angelsmile)\ **Post date:** [February 18, 2016, 11:28pm UTC](https://discuss.elastic.co/t/replica-shards-are-sometimes-not-created/41451/5 "2016-02-18T23:28:38Z")
I changed the number of primary shards to 2 and minimum\_master\_nodes to 2. All replica shards have been created correctly since then. It seems the problem has been solved. Thanks for your help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:15pm UTC](https://discuss.elastic.co/t/replica-shards-are-sometimes-not-created/41451/6 "2017-07-05T23:15:05Z")