# Red cluster and don't come green all the night **URL:** \ **Category:** Elasticsearch\ **Created:** [May 16, 2017, 9:31am UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892 "2017-05-16T09:31:28Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Post date:** [May 16, 2017, 9:31am UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/1 "2017-05-16T09:31:28Z")
Good day I was install plugin yesterday. with manual about rolling upgrade i do next things: > PUT \_cluster/settings > { > "transient": { > "cluster.routing.allocation.enable": "none" > } > } > POST \_flush/synced > systemctl stop elasticsearch > /usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3 > systemctl start elasticsearch > curl -XPUT localhost:9322/\_cluster/settings -d '{ > "transient": { > "cluster.routing.allocation.enable": "all" > } > }' Before all i do { "number\_of\_replicas" :0 } due to lack of space. > Now i have: > "version" : { > "number" : "5.0.0", > "build\_hash" : "253032b", > "build\_date" : "2016-10-26T04:37:51.531Z", > "build\_snapshot" : false, > "lucene\_version" : "6.2.0" > } ``` cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent backup-itest01log red 4 3 3355 3348 0 0 783 0 - 81.1% heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name 60 82 6 1.01 0.47 0.28 - - kibana01 18 86 9 0.50 0.66 1.00 mdi - node02 56 88 8 0.04 0.40 0.81 mdi * node01 22 91 9 0.15 0.40 0.71 mdi - node03 shards disk.indices disk.used disk.avail disk.total disk.percent node 1118 592.9gb 935gb 813.2gb 1.7tb 53 node02 1119 629.9gb 723gb 1tb 1.7tb 41 node01 1118 631.5gb 964.8gb 783.4gb 1.7tb 55 node03 783 UNASSIGNED ``` And it dont't change all the night. node01(master) and client node(kibana01) i don't touch yet. only node02 and 03 Help me please! ---
**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Post date:** [May 16, 2017, 1:23pm UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/2 "2017-05-16T13:23:28Z")
for all unassigned nodes ``` state,unassigned.reason UNASSIGNED NODE_LEFT ``` ---
**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Post date:** [May 16, 2017, 1:51pm UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/3 "2017-05-16T13:51:56Z")
![](https://us1.discourse-cdn.com/elastic/original/3X/1/e/1e831aefc60eefb674a911835fb4d4d3ba89bfce.png) May i somehow delete shard number 2 without data loss? ---
**Author:** ![pjanzen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pjanzen/32/13756_2.png) [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Post date:** [May 17, 2017, 10:49am UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/4 "2017-05-17T10:49:40Z")
If I am not mistaken you need to install repository-s3 on all ES nodes and restart the cluster. Also if you delete that shard you will loose data (I think). ---
**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Post date:** [May 17, 2017, 11:13am UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/5 "2017-05-17T11:13:09Z")
Thank you for answer! i am afraid to restart master node due to red cluster. and don't know how to assigne problematic shards ---
**Author:** ![pjanzen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pjanzen/32/13756_2.png) [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Post date:** [May 17, 2017, 11:30am UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/6 "2017-05-17T11:30:45Z")
I think when you look at the logs from elasticsearch you will find that ES cannot connect to each other due to config changes. You NEED to install the plugin on all ES nodes. For the restart, the cluster is RED, the worst thing that can happen after a restart is the things start working... As long as you do not do a rm -rf on any dir your data will be safe, it will be a bitch to recover but it will be there... ---
**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Post date:** [May 17, 2017, 2:29pm UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/7 "2017-05-17T14:29:35Z")
i delete some old indices, and turn on replicas. wait now for they assign to nodes, then will restart master node ---
**Author:** ![breezzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/breezzz/32/18245_2.png) [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Post date:** [May 18, 2017, 10:14am UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/8 "2017-05-18T10:14:28Z")
plug-in on all nodes now. shards still unassigned ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 15, 2017, 10:17am UTC](https://discuss.elastic.co/t/red-cluster-and-dont-come-green-all-the-night/85892/9 "2017-06-15T10:17:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.