# Running tasks get terminated when cluster is upgraded from 6.6.2 to 7.10.2 using full cluster restart upgrade approach **URL:** \ **Category:** Elasticsearch\ **Created:** [February 9, 2021, 1:23am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668 "2021-02-09T01:23:33Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![yabhishek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yabhishek/32/82979_2.png) [@yabhishek](https://discuss.elastic.co/u/yabhishek)\ **Post date:** [February 9, 2021, 1:23am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/1 "2021-02-09T01:23:33Z")
Hi I tried to upgrade my elasticsearch cluster running version 6.6.2 to version 7.10.2 using full cluster restart upgrade approach. Before performing upgrade, I had some tasks running as can be seen in the following ouput - ```auto "kXxifenTS-eG4airQq7e4g" : { "name" : "qaes-testesupdadata1", "transport_address" : "10.109.12.217:9300", "host" : "10.109.12.217", "ip" : "10.109.12.217:9300", "roles" : [ "master", "data" ], "attributes" : { "ml.machine_memory" : "4085968896", "ml.max_open_jobs" : "20", "xpack.installed" : "true", "ml.enabled" : "true" }, "tasks" : { "kXxifenTS-eG4airQq7e4g:305558" : { "node" : "kXxifenTS-eG4airQq7e4g", "id" : 305558, "type" : "netty", "action" : "cluster:monitor/tasks/lists[n]", "start_time_in_millis" : 1612777918993, "running_time_in_nanos" : 65061, "cancellable" : false, "parent_task_id" : "dhqNUp-ZScu7KFVPGyREfA:326955", "headers" : { } }, "kXxifenTS-eG4airQq7e4g:281310" : { "node" : "kXxifenTS-eG4airQq7e4g", "id" : 281310, "type" : "transport", "action" : "indices:data/write/update/byquery", "start_time_in_millis" : 1612777692354, "running_time_in_nanos" : 226639681547, "cancellable" : true, "headers" : { } }, ``` So I shutdown the nodes and perform the upgrade and when upgraded nodes restart, they join the cluster, shards recover completely and nodes are in healthy state but these tasks are terminated. Is there a way to get around with this or should I prefer rolling upgrade as my application will have tasks running even after I stop indexing before upgrade. Please help! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 9, 2021, 2:14am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/2 "2021-02-09T02:14:41Z")
Welcome to our community! 😃 Are you asking how to restart the `indices:data/write/update/byquery` task after the nodes have shut down? If so there is no way to do this without re-running the original request, which looks like an update-by query. ---
**Author:** ![yabhishek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yabhishek/32/82979_2.png) [@yabhishek](https://discuss.elastic.co/u/yabhishek)\ **Post date:** [February 9, 2021, 2:45am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/3 "2021-02-09T02:45:56Z")
Thanks @warkolm. Would rolling upgrade ensure that such tasks on the node being upgraded won't be terminated rather moved to another running node? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 9, 2021, 2:46am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/4 "2021-02-09T02:46:35Z")
I guess that depends on what the job was doing, but in theory that should work. ---
**Author:** ![yabhishek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yabhishek/32/82979_2.png) [@yabhishek](https://discuss.elastic.co/u/yabhishek)\ **Post date:** [February 9, 2021, 2:48am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/5 "2021-02-09T02:48:15Z")
I will have many indexing, add\_tags and query operations running while I would be upgrading a node. Would those operations be affected by the rolling upgrade although documentation says they shouldn't be? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 9, 2021, 2:49am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/6 "2021-02-09T02:49:07Z")
They shouldn't be as long as you upgrade a node at a time, and then wait for the cluster to be green before moving to the next node. ---
**Author:** ![yabhishek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yabhishek/32/82979_2.png) [@yabhishek](https://discuss.elastic.co/u/yabhishek)\ **Post date:** [February 9, 2021, 2:50am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/7 "2021-02-09T02:50:19Z")
Thanks @warkolm that helps. I will try that on a test cluster before upgrading the production env and see what the behaviour is. Thanks! ---
**Author:** ![yabhishek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yabhishek/32/82979_2.png) [@yabhishek](https://discuss.elastic.co/u/yabhishek)\ **Post date:** [February 10, 2021, 9:26am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/8 "2021-02-10T09:26:39Z")
@warkolm So we thought of going the rolling upgrade way but just wanted to test if shutting down a node would terminate the running merge/add\_tag operations and found that the operations were terminated. So rolling upgrade doesn't seem to solving the problem. Is there a way to force move tasks from one node to another before shutting down the node? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 10, 2021, 9:27am UTC](https://discuss.elastic.co/t/running-tasks-get-terminated-when-cluster-is-upgraded-from-6-6-2-to-7-10-2-using-full-cluster-restart-upgrade-approach/263668/9 "2021-03-10T09:27:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.