# Where should I move proxy nodes in ECE? **URL:** \ **Category:** Elastic Cloud Enterprise (ECE)\ **Created:** [August 5, 2019, 5:54pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860 "2019-08-05T17:54:55Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Post date:** [August 5, 2019, 5:54pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/1 "2019-08-05T17:54:55Z")
I just noticed Kibana was kind of slow in my current deployment, and I wasn't sure why. Maybe, I should have moved proxy nodes from coordinators to allocators. Any thoughts? If I don't want to deploy a dedicate node/s for proxies, where should I keep proxies in the ECE deployment? Should I move them to allocators or should they stay as they are (on coordinators)? I was under impression that nothing should have been moved into allocators for the security reasons. --Thanks ---
**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\ **Post date:** [August 5, 2019, 6:58pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/2 "2019-08-05T18:58:41Z")
We do recommend moving proxy nodes off allocators where possible, either onto the coordinator nodes or onto their own dedicated nodes. This is both security/isolation and performance reasons. (See [https://www.elastic.co/guide/en/cloud-enterprise/current/ece-playbook.html](https://www.elastic.co/guide/en/cloud-enterprise/current/ece-playbook.html)) That said many smaller ECE deployments do co-locate proxies on allocators. It is unlikely that this would be the root cause of a slow Kibana unless there was some anomalous traffic flow (eg large numbers of requests that the proxy was rejecting for some reason). You can use `docker stats` to confirm that the proxy is not CPU-limiting the allocator box, I'd be somewhat surprised if that was the case. Have you looked at the monitoring information (either what ECE generates natively or ideally with a separate monitoring cluster)? Most of the time a slow Kibana is caused by a slow Elasticsearch. ---
**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Post date:** [August 5, 2019, 7:53pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/3 "2019-08-05T19:53:08Z")
I haven't moved proxies back to allocators (since the installation process), I was just thinking about it since allocators are much more powerful than coordinators in my deployment (assuming in any others as well), but again I had concerns that this wasn't a recommendation by Elastic. I checked all the monitoring information dashboards and logs, and everything was green. The usage was super low, no obvious performance issues were showing. The issue I noticed is that sometimes (when you run through the tabs on the left side menu) Kibana shows the first few tabs right away, and then it takes about 5-10 seconds to show the next one you clicked on. And this is constantly happening. Or sometimes when you refresh, it takes about 5 seconds to do it. The other times, it's right away. @Alex_Piggott I ran docker stats commend and noticed frc-proxies-proxy is about 10-20% on the average, but when I keep going through the tabs in Kibana and I get the 'spinning' logo of Kibana, the CPU spikes to 80-90%. The server is pretty much idle, just some test data, and it has 8CPUs and 32GB RAM. ---
**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Post date:** [August 5, 2019, 7:58pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/4 "2019-08-05T19:58:27Z")
Also, after upgrading ECE, I can see some notification about proxies running on coordinators now. Runner [x.x.x.x](https://xxxxxxxxxxx:12443/region/ece-region/runners/x.x.x.x) is both a coordinator and a proxy From your documentation: Roles that should not be held by the same runner: - Allocators and coordinators - Allocators and directors - Coordinators and proxies ---
**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\ **Post date:** [August 5, 2019, 8:59pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/5 "2019-08-05T20:59:59Z")
I'd look at the proxy logs in the L+M cluster and correlate response times with when you see slow down. If you see long response times and no suggestion that ES is slow then maybe a slow coordinator is acting as bottleneck. I'd be really surprised if that could be responsible for a 10s delay. Sounds more like an IO stall on the allocator or connectivity issues or something like that. > Runner [x.x.x.x](https://xxxxxxxxxxx:12443/region/ece-region/runners/x.x.x.x) is both a coordinator and a proxy That's our "best practice" recommendation but lots of people run eg 3 or 6 host ECE deployments and co-locating either allocator/proxy or coordinator/proxy (or both) depending on hardware/expected load etc is super standard. ---
**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\ **Post date:** [August 5, 2019, 9:02pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/6 "2019-08-05T21:02:12Z")
> [@daniel\_a](#): > > I ran docker stats commend and noticed frc-proxies-proxy is about 10-20% on the average, but when I keep going through the tabs in Kibana and I get the 'spinning' logo of Kibana, the CPU spikes to 80-90%. Sorry I missed this. Can I check I understood correctly ... the CPU usage of `frc-proxies-proxy` goes up to 80-90% when you spin through the tabs in Kibana?! How many cores do the coordinator hosts have? (and what sort of volume of traffic does the proxy logs show when this occurs)? ---
**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Post date:** [August 5, 2019, 10:55pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/7 "2019-08-05T22:55:50Z")
@Alex_Piggott We run the whole infra in GCP on SSD drives. I'll look into the proxy logs. ---
**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Post date:** [August 5, 2019, 10:58pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/8 "2019-08-05T22:58:37Z")
Coordinators: 3 systems x 8 vCPUs, 32 GB memory + SSD storage Allocators: 3 systems x 32 vCPUs, 128 GB memory + SSD storage And we're getting barely any data, we just started. ---
**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Post date:** [August 5, 2019, 11:05pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/9 "2019-08-05T23:05:14Z")
Searches return pretty quickly, but the issue we have is with going through the tabs (ex: Discover, Visualize, etc), they're taking 10s+ after you click on a few. Maybe that's a normal behavior, maybe these are apps they're loading a bit longer. Would that be correct? ---
**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\ **Post date:** [August 6, 2019, 12:50pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/10 "2019-08-06T12:50:28Z")
> [@daniel\_a](#): > > Searches return pretty quickly, but the issue we have is with going through the tabs (ex: Discover, Visualize, etc), they're taking 10s+ after you click on a few. Maybe that's a normal behavior, maybe these are apps they're loading a bit longer. Would that be correct? The first time maybe? I wouldn't expect it to happen every time, and the proxy spiking in CPU is a bit odd. The coordinator box you've spec'd looks plenty fast enough that putting the proxy either there or on the allocator shouldn't matter ---
**Author:** ![daniel\_a](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_a/32/48390_2.png) [@daniel\_a](https://discuss.elastic.co/u/daniel_a)\ **Post date:** [August 6, 2019, 4:07pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/11 "2019-08-06T16:07:17Z")
I'll keep digging around and let you know if there is anything odd anywhere in the deployment. --Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 20, 2019, 4:07pm UTC](https://discuss.elastic.co/t/where-should-i-move-proxy-nodes-in-ece/193860/12 "2019-08-20T16:07:19Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.