Getting permission on first Jupyter Lab environment spawn

Hello,

I have recently deployed a JupyterHub on my Kubernetes cluster using the helm chart recommended by Zero-to-JupyterHub present here:

I am also using few custom built images in the JupyterHub for spawning the user environments.

I am facing a weird problem where I am getting “permission denied” error while creating any image/folder at the path “/home/jovyan/work” (where my PVC is mounted) in any spawned image for any user for the first time but it works totally fine for all subsequent spawns for the same user.

As per my observation this behavior is happening when the PVC is created for a particular user while spawning for the first time. It works after every first spawn. If I delete the PVC for that user and spawn it again, it’ll again give the same problem for the first time but will work fine for all subsequent spawns.

Can anyone please help me in resolving this issue?

This sounds like a problem with how permissions are being set by your Kubernetes dynamic volume provisioner. Maybe it’s very slow to set the permissions on the volume? I think you’ll need to research the documentation for your k8s cluster and volume provisioner.