# Backoff after failed scale up
**URL:** https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331
**Category:** JupyterHub
**Created:** [February 13, 2020, 11:03pm UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331 "2020-02-13T23:03:14Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@arokem](https://discourse.jupyter.org/u/arokem)
#### Post date: [February 13, 2020, 11:03pm UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/1 "2020-02-13T23:03:14Z")
Hello! I am using hubploy + pangeo to deploy a jupyterhub on GCP. After one user is logged into the hub and is using substantial resources, I try to log in, but keep running into this message:
and then the spawn times out and fails. During the attempted spawn, sometimes an additional machine will be added to the cluster, but it apparently can’t be used, because I never manage to get in there. Trying `kubectl logs jupyter-arokem -n l2lhub-prod` gives me nothing back. Might this have something to do with the pod affinities? How do I go about mucking with that to fix it? Thanks!
---
### Author:  [@betatim](https://discourse.jupyter.org/u/betatim)
#### Post date: [February 17, 2020, 6:17am UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/2 "2020-02-17T06:17:18Z")
Can you point us to the helm charts + config you are using?
“scale up” and auto-scaling also depends on how you configured the node pools in your GKE cluster, so that config is also needed.
Off the top of my head I don’t know when this error occurs :-/
---
### Author:  [@arokem](https://discourse.jupyter.org/u/arokem)
#### Post date: [February 19, 2020, 6:47am UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/3 "2020-02-19T06:47:42Z")
The configuration for this hub is all in this repo: [https://github.com/learning-2-learn/l2lhub-deployment](https://github.com/learning-2-learn/l2lhub-deployment)
How do I get the GKE config?
Thanks!
---
### Author:  [@betatim](https://discourse.jupyter.org/u/betatim)
#### Post date: [February 19, 2020, 7:07am UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/4 "2020-02-19T07:07:00Z")
After a quick look I can’t spot something obviously wrong in the config.
Which commands did you run to setup your GKE cluster and node pools? I don’t think there is a way to export it so those are probably the best we have.
As you are using hubploy it is probably worth getting @yuvipanda involved.
---
### Author:  [@yuvipanda](https://discourse.jupyter.org/u/yuvipanda)
#### Post date: [February 19, 2020, 7:41am UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/5 "2020-02-19T07:41:42Z")
In the node config for your GKE cluster, there should be a ‘autoscaling -\> max number of nodes’. Can you check what that is?
I think the ‘1 max node group size reached’ is the important part, not the backoff.
---
### Author:  [@arokem](https://discourse.jupyter.org/u/arokem)
#### Post date: [February 19, 2020, 5:52pm UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/6 "2020-02-19T17:52:38Z")
I am not sure where to find that particular thing, but does this answer your question?
---
### Author:  [@minrk](https://discourse.jupyter.org/u/minrk)
#### Post date: [February 21, 2020, 12:21pm UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/7 "2020-02-21T12:21:50Z")
My first hunch is to [check your quotas](https://console.cloud.google.com/iam-admin/quotas). It’s possible that you are using up your CPU or memory quota so scale-up is failing because the next node would exceed some quota.
---
### Author:  [@arokem](https://discourse.jupyter.org/u/arokem)
#### Post date: [February 21, 2020, 1:34pm UTC](https://discourse.jupyter.org/t/backoff-after-failed-scale-up/3331/8 "2020-02-21T13:34:36Z")
Thanks! That is a very good hunch. Indeed, this cluster used to be in another zone, which had the CPU quota set much higher. Now, in a zone with a rather limited quota, which might explain.