# Hub/proxy node affinity

**URL:** https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849
**Category:** JupyterHub
**Created:** [February 2, 2022, 12:41am UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849 "2022-02-02T00:41:16Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![satra](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/satra/32/1458_2.png) [@satra](https://discourse.jupyter.org/u/satra)
#### Post date: [February 2, 2022, 12:41am UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849/1 "2022-02-02T00:41:16Z")

</div>

i’m running two jupyterhub instances on aws with identical settings. they both use cluster autoscaler and have three autoscaling groups: one ondemand and two with spot pricing. all groups have a `minSize` of 0.

in one of the instances the hub and proxy have ended up a spot pricing worker. i would like for them to stay on the ondemand group. is there a known way of forcing the hub/proxy to stay in a certain nodetype?

[Here](https://github.com/ABCD-ReproNim/reprohub/tree/f1011fbffd3ff384e3a83ade4870e5f8f065895d) is the full deployment setup if anyone is interested. The configuration information for one instance is [here](https://github.com/ABCD-ReproNim/reprohub/tree/f1011fbffd3ff384e3a83ade4870e5f8f065895d/dandi-info).

---

<div class="post-metadata">

### Author: ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)
#### Post date: [February 2, 2022, 2:54pm UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849/2 "2022-02-02T14:54:26Z")

</div>

Have you looked at using a `nodeSelector`?  
[https://zero-to-jupyterhub.readthedocs.io/en/latest/resources/reference.html?highlight=nodeselector#hub](https://zero-to-jupyterhub.readthedocs.io/en/latest/resources/reference.html?highlight=nodeselector#hub)

> **[Assigning Pods to Nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/)**
>
> You can constrain a Pod so that it can only run on particular set of Node(s). There are several ways to do this and the recommended approaches all use label selectors to facilitate the selection. Generally such constraints are unnecessary, as the...

---

<div class="post-metadata">

### Author: ![consideRatio](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/consideratio/32/1033_2.png) [@consideRatio](https://discourse.jupyter.org/u/consideRatio)
#### Post date: [February 2, 2022, 4:40pm UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849/3 "2022-02-02T16:40:14Z")

</div>

I recommend configuring `scheduling.userPods.nodeAffinity.matchNodePurpose=require`, like that, you can force user pods and user placeholder pods to schedule on nodes labelled like: `hub.jupyter.org/node-purpose=user`.

With that said, there is a bug reported about this, so I’m not 100% confident this works as intended for the user pods: [(config typo) Are the docs regarding dedicated node pools correct? · Issue #2561 · jupyterhub/zero-to-jupyterhub-k8s · GitHub](https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/2561)

### Related

- [Optimizations — Zero to JupyterHub with Kubernetes documentation](https://zero-to-jupyterhub.readthedocs.io/en/latest/administrator/optimization.html)
- [Configuration Reference — Zero to JupyterHub with Kubernetes documentation](https://zero-to-jupyterhub.readthedocs.io/en/latest/resources/reference.html?highlight=nodeselector#scheduling-userpods-nodeaffinity-matchnodepurpose)

---

<div class="post-metadata">

### Author: ![satra](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/satra/32/1458_2.png) [@satra](https://discourse.jupyter.org/u/satra)
#### Post date: [February 2, 2022, 5:48pm UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849/4 "2022-02-02T17:48:03Z")

</div>

thank @manics . just to clarify. if for one of my node spec’s i add those labels and taints from the core, is that sufficient. i’m listing all the different labels that exists in the spec for different parts. is `metadata --> labels` and `spec --> taints` the correct places?

```auto
metadata:
  labels:
    kops.k8s.io/cluster: {{ namespace }}.k8s.local
    hub.jupyter.org/node-purpose: core
  name: nodes-us-east-2a
spec:
  cloudLabels:
    k8s.io/cluster-autoscaler/enabled: ""
    k8s.io/cluster-autoscaler/{{ namespace }}.k8s.local: ""
    k8s.io/cluster-autoscaler/node-template/label: ""
...
  nodeLabels:
    on-demand: "true"
...
  taints:
  - on-demand=true:PreferNoSchedule
  - hub.jupyter.org/dedicated=core:NoSchedule

```

---

<div class="post-metadata">

### Author: ![dhirschfeld](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/dhirschfeld/32/26_2.png) [@dhirschfeld](https://discourse.jupyter.org/u/dhirschfeld)
#### Post date: [February 3, 2022, 5:00am UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849/5 "2022-02-03T05:00:59Z")

</div>

It appears that the _“bug”_ was just user-error 😅

---

<div class="post-metadata">

### Author: ![satra](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/satra/32/1458_2.png) [@satra](https://discourse.jupyter.org/u/satra)
#### Post date: [February 3, 2022, 3:49pm UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849/6 "2022-02-03T15:49:10Z")

</div>

just a note that after i put the label and taints in, the kubernetes cluster would not finish applying a rolling update. i suspect i’m missing something something else. will continue investigating.

---

<div class="post-metadata">

### Author: ![satra](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/satra/32/1458_2.png) [@satra](https://discourse.jupyter.org/u/satra)
#### Post date: [February 12, 2022, 2:37am UTC](https://discourse.jupyter.org/t/hub-proxy-node-affinity/12849/7 "2022-02-12T02:37:03Z")

</div>

just an update here that i had to change some of the node taints independent of jupyterhub and increase the number of nodes. this keeps the hub/proxy pods on these lighterweight nodes, allowing the spot nodes to spin down. specifically i was setting `on-demand=False:preferNoSchedule` on these nodes.
