Cannot spawn server for new user

JupyterHub Zero to JupyterHub on Kubernetes
21

Ok will try right now and get back to you

22

Doesn’t work. No progress at all. ends up failing

23

PVC Logs:

kubectl describe pvc claim-zfjryuibwedbvgjvmxrvjcfjybevht -n jupyterhub
Name:          claim-zfjryuibwedbvgjvmxrvjcfjybevht
Namespace:     jupyterhub
StorageClass:  standard-rwo
Status:        Pending
Volume:        
Labels:        app=jupyterhub
               chart=jupyterhub-3.3.6
               component=singleuser-storage
               heritage=jupyterhub
               hub.jupyter.org/username=zfjryuibwedbvgjvmxrvjcfjybevht
               release=3.3.6
Annotations:   hub.jupyter.org/username: zfjryuibwedbvgjvmxrvjcfjybevht
               volume.beta.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
               volume.kubernetes.io/selected-node: gke-jupyterhub-staging-default-pool-1a1c895d-5fc9
               volume.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       jupyter-zfjryuibwedbvgjvmxrvjcfjybevht
Events:
  Type    Reason                Age                       From                                                                                              Message
  ----    ------                ----                      ----                                                                                              -------
  Normal  Provisioning          2m59s (x1858 over 4d21h)  pd.csi.storage.gke.io_gke-9083b8f6bc1e4aee93de-001a-abd5-vm_0106cbe6-800c-489c-805c-b17e8e7ff31d  External provisioner is provisioning volume for claim "jupyterhub/claim-zfjryuibwedbvgjvmxrvjcfjybevht"
  Normal  ExternalProvisioning  46s (x28126 over 4d21h)   persistentvolume-controller                                                                       Waiting for a volume to be created either by the external provisioner 'pd.csi.storage.gke.io' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered
24

People online are saying its a storage class issue. How do I find out if it is

25

BTW More logs:

kubectl get sc -n jupyterhub
NAME                     PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
premium-rwo              pd.csi.storage.gke.io   Delete          WaitForFirstConsumer   true                   5d19h
standard                 kubernetes.io/gce-pd    Delete          Immediate              true                   5d19h
standard-rwo (default)   pd.csi.storage.gke.io   Delete          WaitForFirstConsumer   true                   5d19h
26

Disable the user scheduler and see if it works with the gke default kube-scheduler doing the scheduling. See z2jh.jupyter.org and the configuration reference on hiw, i think scheduling.userScheduler.enabled=false

27

Same thing. It didn’t work. PVC logs are the same still(and it fails to spawn server)

28

This is the default storage class btw:

kubectl describe sc standard-rwo
Name:                  standard-rwo
IsDefaultClass:        Yes
Annotations:           components.gke.io/component-name=pdcsi,components.gke.io/component-version=0.18.7,components.gke.io/layer=addon,storageclass.kubernetes.io/is-default-class=true
Provisioner:           pd.csi.storage.gke.io
Parameters:            type=pd-balanced
AllowVolumeExpansion:  True
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     WaitForFirstConsumer
Events:                <none>
29

Fixed by switching to autoprovisioning