# Waiting for nodes to stop - timeout **URL:** \ **Category:** Elastic Cloud Enterprise (ECE)\ **Created:** [May 1, 2017, 7:25am UTC](https://discuss.elastic.co/t/waiting-for-nodes-to-stop-timeout/84134 "2017-05-01T07:25:15Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![iremmats](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iremmats/32/55316_2.png) [@iremmats](https://discuss.elastic.co/u/iremmats)\ **Post date:** [May 1, 2017, 7:25am UTC](https://discuss.elastic.co/t/waiting-for-nodes-to-stop-timeout/84134/1 "2017-05-01T07:25:15Z")
While resizing a three node cluster it doesn't show as completed action due to timeout when stopping nodes. Change memory per node from 4 GB to 8 GB Timeout for configuration change: 5h 33m Unexpected error during step: [wait-until-stopped]: [no.found.constructor.models.TimeoutException: Timeout] Starting step: [wait-until-stopped]: instances=List(ElasticsearchInstance(ElasticsearchCluster(d2d1c12716a04716baadc3e6795d2ecc),instance-0000000011), ElasticsearchInstance(ElasticsearchCluster(d2d1c12716a04716baadc3e6795d2ecc),instance-0000000010), ElasticsearchInstance(ElasticsearchCluster(d2d1c12716a04716baadc3e6795d2ecc),instance-0000000009)) The new nodes show up perfectly fine and it works to index data and search. What should I do here? Kill the docker containers manually? ![](https://us1.discourse-cdn.com/elastic/original/3X/3/4/34d0336d91f0236ed0b796a99de0ae483fa8fbc3.png) ---
**Author:** ![uricohen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/uricohen/32/9776_2.png) [@uricohen](https://discuss.elastic.co/u/uricohen)\ **Post date:** [May 1, 2017, 9:10am UTC](https://discuss.elastic.co/t/waiting-for-nodes-to-stop-timeout/84134/2 "2017-05-01T09:10:23Z")
Hi @iremmats This looks like a Docker related issue. Which version of ECE are you running? In addition, can you please do the following: 1. Fetch all the logs from the relevant allocator hosts and post them here. Use this command to zip them on each host: `docker ps -a > /mnt/data/elastic/docker.out && tar czvf ece-logs.tgz $(find /mnt/data/elastic -name "*.log" -o -name "*.out")` 2. Try to resume the stopped instances, and then stop them manually. 3. If that doesn't help, kill the old 4GB containers in each host (the relevant containers name are of the form `fac-{cluster-id}-{instance-id}`. 4. If the plan is not running (failed because of a timeout), try to resubmit it so it cleans up the old instances. In any case, would be good to see those logs so we can further investigate. HTH, Uri ---
**Author:** ![iremmats](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iremmats/32/55316_2.png) [@iremmats](https://discuss.elastic.co/u/iremmats)\ **Post date:** [May 1, 2017, 5:38pm UTC](https://discuss.elastic.co/t/waiting-for-nodes-to-stop-timeout/84134/3 "2017-05-01T17:38:59Z")
We are running Beta2. 1. I got the logs. They are linked to my dropbox. 2. I resumed the instances in the admin console and then stopped them in the admin console. Seems one of the allocators wasn't able to stop the container. 3. Both stop and kill the container did not work. I found a few similar docker-related issues. [https://github.com/moby/moby/issues/208](https://github.com/moby/moby/issues/208) I also tried attaching to the container but nothing happens. > **[Dropbox - Link not found](https://www.dropbox.com/s/xwib6mczh0vktx2/logs1.tgz?dl=0)** > > Dropbox is a free service that lets you bring your photos, docs, and videos anywhere and share them easily. Never email yourself a file again! > **[Dropbox - Link not found](https://www.dropbox.com/s/bqe2tphqbdfmgq1/logs2.tgz?dl=0)** > > Dropbox is a free service that lets you bring your photos, docs, and videos anywhere and share them easily. Never email yourself a file again! > **[Dropbox - Link not found](https://www.dropbox.com/s/xr66iru4w78vet0/logs3.tgz?dl=0)** > > Dropbox is a free service that lets you bring your photos, docs, and videos anywhere and share them easily. Never email yourself a file again! From our perspective as potential ECE customers the docker handling and docker itself is super important to work flawlessly. ---
**Author:** ![iremmats](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iremmats/32/55316_2.png) [@iremmats](https://discuss.elastic.co/u/iremmats)\ **Post date:** [May 1, 2017, 5:44pm UTC](https://discuss.elastic.co/t/waiting-for-nodes-to-stop-timeout/84134/4 "2017-05-01T17:44:32Z")
2637b394b726 [docker.elastic.co/cloud-enterprise/elasticsearch:5.3.0-1](http://docker.elastic.co/cloud-enterprise/elasticsearch:5.3.0-1) "/sbin/entry-point" 4 days ago Restarting (1) 30 hours ago 0.0.0.0:18438-\>18438/tcp, 0.0.0.0:19296-\>19296/tcp fac-d2d1c12716a04716baadc3e6795d2ecc-instance-0000000010 It seems to be stuck in some restarting state. ---
**Author:** ![iremmats](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iremmats/32/55316_2.png) [@iremmats](https://discuss.elastic.co/u/iremmats)\ **Post date:** [May 1, 2017, 7:45pm UTC](https://discuss.elastic.co/t/waiting-for-nodes-to-stop-timeout/84134/5 "2017-05-01T19:45:06Z")
A restart of the server solved the docker issue. Would be interesting to know what caused it though. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 15, 2017, 7:58pm UTC](https://discuss.elastic.co/t/waiting-for-nodes-to-stop-timeout/84134/6 "2017-05-15T19:58:42Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.