# Waiting for admin cluster to become reachable, after OS upgrade from rhek 7.6 to rhel 7.7

**URL:** <https://discuss.elastic.co/t/waiting-for-admin-cluster-to-become-reachable-after-os-upgrade-from-rhek-7-6-to-rhel-7-7/207343>\
**Category:** Elastic Cloud Enterprise (ECE)\
**Created:** [November 11, 2019, 11:30am UTC](https://discuss.elastic.co/t/waiting-for-admin-cluster-to-become-reachable-after-os-upgrade-from-rhek-7-6-to-rhel-7-7/207343 "2019-11-11T11:30:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 11, 2019, 11:30am UTC](https://discuss.elastic.co/t/waiting-for-admin-cluster-to-become-reachable-after-os-upgrade-from-rhek-7-6-to-rhel-7-7/207343/1 "2019-11-11T11:30:53Z")

</div>

Hi there,

I had to upgrade the OS and for this I deleted the ECE completely like mentioned in the docs. It worked and I reinstalled and rejoined the node. In the UI I can see the runner and configure the roles. But it seems that it has an issue with the admin cluster and the Proxy.

```auto
 [2019-11-11 11:17:06,115][WARN][spray.can.client.HttpHostConnectionSlot] Connection attempt to containerhost:9244 failed in response to POST request to /allocator-metricbeat-*/_search with no retries left, dispatching error... {}
[2019-11-11 11:17:06,549][WARN][spray.can.client.HttpHostConnectionSlot] Connection attempt to containerhost:9244 failed in response to GET request to /.migration/doc/lock with no retries left, dispatching error... {}
[2019-11-11 11:17:06,549][INFO][no.found.adminconsole.elasticsearch.IndexConfigurationActor] Waiting for admin cluster to become reachable ([Connection attempt to containerhost:9244 failed]). Retrying every [5 seconds]. {}
spray.can.Http$ConnectionAttemptFailedException: Connection attempt to containerhost:9244 failed
        at spray.can.client.HttpHostConnectionSlot$$anonfun$connecting$1.applyOrElse(HttpHostConnectionSlot.scala:87)
        at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
        at spray.can.client.HttpHostConnectionSlot.aroundReceive(HttpHostConnectionSlot.scala:33)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
        at akka.actor.ActorCell.invoke(ActorCell.scala:495)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
        at akka.dispatch.Mailbox.run(Mailbox.scala:224)
        at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

```

It is not possible to expand the admin console to the host. Other deployments are working.  
And I cannot Login with its IP:12433.

On this hosts I have added IPtables rules:

```auto
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere multiport dports 12191:12201 /* 900 profile::linux::firewall::deny_unused accept 12191-12201 */
ACCEPT tcp -- anywhere anywhere multiport dports 12343 /* 900 profile::linux::firewall::deny_unused accept 12343 */
ACCEPT tcp -- anywhere anywhere multiport dports 12443 /* 900 profile::linux::firewall::deny_unused accept 12443 */
ACCEPT tcp -- anywhere anywhere multiport dports 12898:12908 /* 900 profile::linux::firewall::deny_unused accept 12898-12908 */
ACCEPT tcp -- anywhere anywhere multiport dports 13898:13908 /* 900 profile::linux::firewall::deny_unused accept 13898-13908 */
ACCEPT tcp -- anywhere anywhere multiport dports 18000:21999 /* 900 profile::linux::firewall::deny_unused accept 18000-21999 */
ACCEPT tcp -- anywhere anywhere multiport dports 2112/* 900 profile::linux::firewall::deny_unused accept 2112 */
ACCEPT tcp -- anywhere anywhere multiport dports ssh /* 900 profile::linux::firewall::deny_unused accept 22 */
ACCEPT tcp -- anywhere anywhere multiport dports 22191:22195 /* 900 profile::linux::firewall::deny_unused accept 22191-22195 */
ACCEPT tcp -- anywhere anywhere multiport dports 9243 /* 900 profile::linux::firewall::deny_unused accept 9243 */
ACCEPT tcp -- anywhere anywhere multiport dports 9343/* 900 profile::linux::firewall::deny_unused accept 9343 */

```

Does anyone has experience with this?  
Greetings  
Malte

---

<div class="post-metadata">

**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\
**Post date:** [November 11, 2019, 7:17pm UTC](https://discuss.elastic.co/t/waiting-for-admin-cluster-to-become-reachable-after-os-upgrade-from-rhek-7-6-to-rhel-7-7/207343/2 "2019-11-11T19:17:14Z")

</div>

There's 2 possible candidate issues here:

1. the `9244` described by the error is managed by a container called `frc-services-forwarders-services-forwarder` - is that container running / healthy?
2. Can you check that `containerhost` is mapped to the right IP address, eg `docker -it frc-admin-consoles-admin-console host containerhost` (and then check if 9244 is bound to the wrong host for some reason?)

---

<div class="post-metadata">

**Author:** ![logger](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@logger](https://discuss.elastic.co/u/logger)\
**Post date:** [November 12, 2019, 7:49am UTC](https://discuss.elastic.co/t/waiting-for-admin-cluster-to-become-reachable-after-os-upgrade-from-rhek-7-6-to-rhel-7-7/207343/3 "2019-11-12T07:49:32Z")

</div>

I checked it, but I think there is something more wrong in my Installation.

I will start a new one and try it step by step.

I found out there are a lot more errors. I will try a new one and if there are still errors I will come back.

Thanks  
Malte

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 26, 2019, 7:49am UTC](https://discuss.elastic.co/t/waiting-for-admin-cluster-to-become-reachable-after-os-upgrade-from-rhek-7-6-to-rhel-7-7/207343/4 "2019-11-26T07:49:47Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
