Ensuring single user pods placement to nodes

JupyterHub Zero to JupyterHub on Kubernetes
1

I have a scenario where I need user pods having a specific label (component=singleuser-server for example) to be deployed on a single node, ensuring that no more than one pod with this specific label is present on any given node.

i.e if 10 users pods are running concurrently, then each pod to be running on a seperate node.

Any ideas on how could I achieve this?

1 Like
2

This is more a k8s related question than JupyterHub. Maybe topology constraints is what you are looking for?

1 Like
3

Required pod anti-affinity can work, or requesting sufficient amount of resources so only one fit per node, or the topology constraints also - ive never used those myself yet.

2 Likes
4

Thank you both for your suggestions! I tried the below configuration of topology constraints and it seems to work!

profileList:
    - display_name: "Low"
      description: "Datascience Environment with Sample Notebooks"
      kubespawner_override:
      image: jupyter/datascience-notebook:ubuntu-22.04
        extra_pod_config:
          {
            "affinity": # This schedules pods to run on the given instance types
              {
                "nodeAffinity":
                  {
                    "requiredDuringSchedulingIgnoredDuringExecution":
                      {
                        "nodeSelectorTerms":
                          [
                            {
                              "matchExpressions":
                                [
                                  {
                                    "key": "node.kubernetes.io/instance-type",
                                    "operator": "In",
                                    "values": ["t3a.xlarge"],
                                  },
                                ],
                            },
                          ],
                      },
                  },
              },
            "topologySpreadConstraints": # This guarantees that exactly one user pod is being scheduled on each node.          [
                {
                  "maxSkew": 1,
                  "topologyKey": "kubernetes.io/hostname",
                  "whenUnsatisfiable": "DoNotSchedule",
                  "labelSelector":
                    {
                      "matchLabels":
                        {
                          "component" : "singleuser-server"
                        },
                    },
                },
              ],
          }
2 Likes
5

Cheers for posting the config here @agelosnm

Would be really useful for others!

2 Likes
6

Note that for something as straight forward as configured in the affinity section, you can also use a nodeSelector (node_selector in kubespawner config), and don’t have to use extra_pod_config for that part.

See Assigning Pods to Nodes | Kubernetes

1 Like