# Shards unassigned after elasticsearch restart **URL:** \ **Category:** Elasticsearch\ **Created:** [August 5, 2017, 10:45pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994 "2017-08-05T22:45:45Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 5, 2017, 10:45pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/1 "2017-08-05T22:45:46Z")
My elasticsearch instances were restarted, but it looks like after that I have about 10 unassisned shards ☹ The reason seems to be node\_left, how can I re-assign the shards to the respective nodes? { "cluster\_name" : "santorini\_rec", "status" : "yellow", "timed\_out" : false, "number\_of\_nodes" : 5, "number\_of\_data\_nodes" : 3, "active\_primary\_shards" : 16, "active\_shards" : 18, "relocating\_shards" : 0, "initializing\_shards" : 4, "unassigned\_shards" : 10, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 2, "number\_of\_in\_flight\_fetch" : 0, "task\_max\_waiting\_in\_queue\_millis" : 0, "active\_shards\_percent\_as\_number" : 56.25 } Does this mean I lost all the data? .monitoring-es-6-2017.08.05 0 r UNASSIGNED NODE\_LEFT santo\_pipeline\_data 5 r UNASSIGNED NODE\_LEFT santo\_pipeline\_data 3 r UNASSIGNED NODE\_LEFT santo\_pipeline\_data 8 r UNASSIGNED NODE\_LEFT santo\_pipeline\_data 2 r UNASSIGNED NODE\_LEFT santo\_pipeline\_data 1 r UNASSIGNED PRIMARY\_FAILED .monitoring-alerts-6 0 r UNASSIGNED NODE\_LEFT .watches 0 r UNASSIGNED NODE\_LEFT .watcher-history-3-2017.08.05 0 r UNASSIGNED NODE\_LEFT .monitoring-kibana-6-2017.08.05 0 r UNASSIGNED NODE\_LEFT I tried to enable re-allocation, but it dint help. { "transient": { "cluster.routing.allocation.enable": "all" } } ' Elasticsearch seems to be so sensitive about any operation. Everytime I touch it, something or the other goes wrong and its just so hard to recover it back. Can someone please help? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 5, 2017, 11:04pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/2 "2017-08-05T23:04:16Z")
What version are you on? ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 5, 2017, 11:05pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/3 "2017-08-05T23:05:47Z")
@warkolm I am on 5.5.1 I just tried turning off all replicas and turning back on again, not sure if it will help. Its still initializing the shards. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 5, 2017, 11:06pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/4 "2017-08-05T23:06:13Z")
Why were the instances restarted? ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 5, 2017, 11:07pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/5 "2017-08-05T23:07:10Z")
I disabled swapping on all hosts, and I had to tune some networking related parameters. So I gracefully shut down elastic using systemd , except for the master node. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 5, 2017, 11:07pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/6 "2017-08-05T23:07:41Z")
Did you shut down all the data nodes at once? ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 5, 2017, 11:07pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/7 "2017-08-05T23:07:58Z")
No, one by one, making sure each instance came up. ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 5, 2017, 11:09pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/8 "2017-08-05T23:09:18Z")
Not sure if it was a good idea, i turned replica to 0 and back to 10 again... "cluster\_name" : "santorini\_rec", "status" : "yellow", "timed\_out" : false, "number\_of\_nodes" : 5, "number\_of\_data\_nodes" : 3, "active\_primary\_shards" : 16, "active\_shards" : 23, "relocating\_shards" : 0, "initializing\_shards" : 6, "unassigned\_shards" : 84, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 0, "number\_of\_in\_flight\_fetch" : 0, "task\_max\_waiting\_in\_queue\_millis" : 0, "active\_shards\_percent\_as\_number" : 20.353982300884958 } ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 5, 2017, 11:16pm UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/9 "2017-08-05T23:16:39Z")
I still see indexing work and the logs are being ingested. Should I shut down logstash until I fix the state, or it should be ok? ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 6, 2017, 1:25am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/10 "2017-08-06T01:25:22Z")
All the unassigned are replicas. Unable to recover them. Appreciate any suggestions. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 6, 2017, 4:07am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/11 "2017-08-06T04:07:57Z")
Why do you need 10 replicas? ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 6, 2017, 4:14am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/12 "2017-08-06T04:14:20Z")
Actually I dont, I created 9 shards for 3 nodes, and it automatically created 10 replicas total. Some of them recovered automatically ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 6, 2017, 4:15am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/13 "2017-08-06T04:15:26Z")
You definitely have more than one replica set, and if you have more than 2 then those extras won't even be assigned. ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 6, 2017, 4:16am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/14 "2017-08-06T04:16:56Z")
Can I reduce the numnber of replicas now? What solution would you suggest ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 6, 2017, 9:12am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/15 "2017-08-06T09:12:25Z")
Usually having more than 1 replica set is not required unless you have unstable infrastructure or low volumes of data queried at a high rate. You have 3 data nodes, that means you can only store the primary and 2 replica sets of the data. Setting more will mean unassigned (replica) shards, which causes the yellow status until you either have more nodes to hold them or you reduce the replica count to fit in your cluster. `curl -XPUT localhost:9200/*/_settings -d '{ "index" : { "number_of_replicas" : N } }'`, where `N` is the number you want. ---
**Author:** ![Drift](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Drift](https://discuss.elastic.co/u/Drift)\ **Post date:** [August 8, 2017, 12:28am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/16 "2017-08-08T00:28:00Z")
Thankyou @warkolm. That makes sense. I have adjusted my replicas accordingly and got it to green now 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 5, 2017, 12:28am UTC](https://discuss.elastic.co/t/shards-unassigned-after-elasticsearch-restart/95994/17 "2017-09-05T00:28:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.