# 7.3.1 After upgrading to 7.6.2, the cluster master freezes irregularly **URL:** \ **Category:** Elasticsearch\ **Created:** [April 21, 2020, 6:43am UTC](https://discuss.elastic.co/t/7-3-1-after-upgrading-to-7-6-2-the-cluster-master-freezes-irregularly/228975 "2020-04-21T06:43:23Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![zcola](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zcola/32/27971_2.png) [@zcola](https://discuss.elastic.co/u/zcola)\ **Post date:** [April 21, 2020, 6:43am UTC](https://discuss.elastic.co/t/7-3-1-after-upgrading-to-7-6-2-the-cluster-master-freezes-irregularly/228975/1 "2020-04-21T06:43:23Z")
> > > Describe the feature: > Elasticsearch version (bin/elasticsearch --version): > 7.6.2 > Plugins installed: \[\] > IK > JVM version (java -version): > 13 > OS version (uname -a if on a Unix-like system): > Debian 8 > Description of... ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 21, 2020, 6:45am UTC](https://discuss.elastic.co/t/7-3-1-after-upgrading-to-7-6-2-the-cluster-master-freezes-irregularly/228975/2 "2020-04-21T06:45:43Z")
Providing a link to a GitHub issue doesn't really clarify what you are expecting here sorry to say. ---
**Author:** ![zcola](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zcola/32/27971_2.png) [@zcola](https://discuss.elastic.co/u/zcola)\ **Post date:** [April 23, 2020, 3:01am UTC](https://discuss.elastic.co/t/7-3-1-after-upgrading-to-7-6-2-the-cluster-master-freezes-irregularly/228975/3 "2020-04-23T03:01:52Z")
There were problems a few hours after the upgrade completed normal operation on the morning of April 9, the specific performance was - Replica shards cannot be allocated, the newly created index has no data, master and logstash both appear `failed to process cluster event (put-mapping) within 30s` related logs - Trigger rollover stuck - Pending task has more than a dozen tasks, and will never disappear, normally empty - Delete index is normal, all nodes have joined the cluster, and node load is normal Restart the elected master After re-election, triggering the re-election of the master is normal, but on average it will freeze once every few hours. Within six months before the upgrade we have not had this problem, there is not much change in the business, 3 dedicated master node, hot node shard number 50 +, stale node shard number 200+. freeze 300+ shards, a total of 62 nodes **Provide logs (if relevant)**: The master log only appears when there is a problem ```auto org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-mapping) within 30s, at org.elasticsearch.cluster.service.MasterService$Batcher.lambda$onTimeout$0(MasterService.java:143) [elasticsearch-7.6.2.jar:7.6.2], at java.util.ArrayList.forEach(ArrayList.java:1507) [?:?], at org.elasticsearch.cluster.service.MasterService$Batcher.lambda$onTimeout$1(MasterService.java:142) [elasticsearch-7.6.2.jar:7.6.2], at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:633) [elasticsearch-7.6.2.jar:7.6.2], at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?], at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?], at java.lang.Thread.run(Thread.java:830) [?:?] o.e.c.r.a.DiskThresholdMonitor skipping monitor as a check is already in progress explaining the allocation for [ClusterAllocationExplainRequest[useAnyUnassignedShard=true,includeYesDecisions?=false], found shard [[push_up_new_log-2020.04.10-000049][1], node[null], [R], recovery_source[peer recovery], s[UNASSIGNED], unassigned_info[[reason=INDEX_CREATED], at[2020-04-10T03:06:20.497Z], delayed=false, allocation_status[no_attempt]]] ``` Execute explain api ```auto { "index": "push_up_new_log-2020.04.10-000049", "shard": 1, "primary": false, "current_state": "unassigned", "unassigned_info": { "reason": "INDEX_CREATED", "at": "2020-04-10T03:06:20.497Z", "last_allocation_status": "no_attempt" }, "can_allocate": "yes", "allocate_explanation": "can allocate the shard", "target_node": { "id": "JLxrX6zoStKFzE6lFsi76w", "name": "data-51-hot", "transport_address": "10.90.141.133:9300", "attributes": { "zone": "hot", "xpack.installed": "true" } }, ``` Execute cluster / pending\_task, kibana\_index\_template can be ignored, there is always, I do n’t know where 6.x kibana connects this es ```auto { "tasks": [ { "insert_order": 34926, "priority": "URGENT", "source": "create-index-template [kibana_index_template:.kibana], cause [api]", "executing": true, "time_in_queue_millis": 489, "time_in_queue": "489ms" }, { "insert_order": 34927, "priority": "URGENT", "source": "create-index-template [kibana_index_template:.kibana], cause [api]", "executing": false, "time_in_queue_millis": 82, "time_in_queue": "82ms" }, { "insert_order": 34928, "priority": "URGENT", "source": "create-index-template [kibana_index_template:.kibana], cause [api]", "executing": false, "time_in_queue_millis": 37, "time_in_queue": "37ms" }, { "insert_order": 34883, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13545, "time_in_queue": "13.5s" }, { "insert_order": 34882, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13546, "time_in_queue": "13.5s" }, { "insert_order": 34888, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13494, "time_in_queue": "13.4s" }, { "insert_order": 34884, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13535, "time_in_queue": "13.5s" }, { "insert_order": 34887, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13494, "time_in_queue": "13.4s" }, { "insert_order": 34897, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 11554, "time_in_queue": "11.5s" }, { "insert_order": 34886, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13523, "time_in_queue": "13.5s" }, { "insert_order": 34889, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13487, "time_in_queue": "13.4s" }, { "insert_order": 34896, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 11554, "time_in_queue": "11.5s" }, { "insert_order": 34885, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 13531, "time_in_queue": "13.5s" }, { "insert_order": 34902, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 11295, "time_in_queue": "11.2s" }, { "insert_order": 34895, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 11792, "time_in_queue": "11.7s" }, { "insert_order": 34898, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 11460, "time_in_queue": "11.4s" }, { "insert_order": 34901, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 11295, "time_in_queue": "11.2s" }, { "insert_order": 34899, "priority": "HIGH", "source": "put-mapping", "executing": false, "time_in_queue_millis": 11458, "time_in_queue": "11.4s" }, { "insert_order": 31866, "priority": "NORMAL", "source": "cluster_reroute(reroute after starting shards)", "executing": false, "time_in_queue_millis": 1212977, "time_in_queue": "20.2m" } ] } ``` 尝试降级提示 ` java.lang.IllegalStateException: cannot downgrade a node from version [7.6.2] to version [7.3.1]",` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 21, 2020, 3:01am UTC](https://discuss.elastic.co/t/7-3-1-after-upgrading-to-7-6-2-the-cluster-master-freezes-irregularly/228975/4 "2020-05-21T03:01:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.