Jupyterhub Pods all going to only one node on the cluster

I think you are correct it is the storage, but I am not able to figure it out. First I tried

singleuser:
  storage:
    type: none

I also tried

singleuser:
  storage:
    dynamic:
      storageClass: ugba88-2-sc

Each time I started up 150 users and they always go to the same node.

I have NFS mounted common storage mounted on all the nodes. So I assume since the storage is “local” I can just point to it.

I defined a storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ugba88-2-sc
provisioner: kubernetes.io/no-provisioner
parameters:
  # The path to the local storage on the node.
  local: /mnt/ist/jhub-stor/2023/fall/ugba88/

Then I defined a pv:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: ugba88-2-pv
spec:
  capacity:
    storage: 300Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: ugba88-2-sc
  local:
    path: /mnt/ist/jhub-stor/2023-fall/ugba88
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - jup2
          - jup3
          - jup4
          - jup5
          - jup6
          - jup7
          - jup8
          - jup9

and I defined a pvc

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ugba88-2-pvc
spec:
  storageClassName: ugba88-2-sc
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

Is there a better way to do this? Many years ago I was able to use HostPath directory in the jupyterhub yaml, that was less complicated :slight_smile: