I am new to jupyterhub (I’m a linux sysadmin, not an end user) and just spun up a server on ubuntu 20.04 and it successfully starts and users can log in, but it fails when they try to launch a notebook. The spawner fails with:
Error in Authenticator.pre_spawn_start: PermissionError [Errno 13] Permission denied: ‘/Users/jdoe/.jupyterhub’
My home directories are on an NFS share. It’s interesting that it successfully creates the .jupyterhub directory but then can’t write to it? Any ideas? Thanks in advance (-:
I have a similar question; I’m deploying JupyterHub with the Helm chart and would like to mount a single already-extant CephFS volume on behalf of the notebook user with a subpath specification to put them in their home directory. Is it possible to run the notebook server and mount the volume with the uid/gid of the authenticated user?
The permissions on the mounted storage are controlled by your K8s storage provider. For example, some storage providers will set the permissions to 0777 allowing read-write for all UIDs in the pod mounting the volume.
Others will chmod the directory the group specified by fsGroup. If your storage provider supports fsGroup you can configure it in Z2JH using singleuser.fsGid.
You’ll have to read up on your K8s storage provider to find out what if anything it supports. If necessary you can create a custom StorageClass, but again this is outside the control of Z2JH.
I’ve set permissions to 077on my test home directory, with the same result. Seems to me if it can successfully create the .jupyterhub directory there is something more subtle going on here? I even then set the perms on the new directory to wide open, but no luck. I use the same NFS server w/ my rstudio pro server and it has no issues creating & modifying what user files it needs.
@manics, I understand what you’re saying about fsGroup, and indeed some storageclass implementations will reach right into a volume to own its contents, but that’s not what I’m after in a multitenant volume; I specifically want a user’s container to run as their UID/GID, and to mount the multitenant “/home” volume that all the platform users have their own subdirectories in with only the privileges of their own identity.
I can see a fairly heavy-handed way to accomplish this with a custom init container, but I had hoped given JupyterHub’s multitenant design that it would have something for this scenario already included.
Hi! In your original post you didn’t say your users’s directories already exist, it sounded like you wanted to create a (sub)volume with custom UID on demand.
If the user’s directory already exists then you can run your server as a different UID/GID, see for example
It’s a bit out of date but the idea still holds, create a pre_spawn_start hook that passes the uid/gid to the singleuser server, run the server as root, and switch to that UID. You might even be able to run as that UID instead of root but I’ve never tried it.
That’s super helpful, thanks! I guess I should say, I can’t guarantee that the user’s home directory will already exist in the common volume - this deployment is still being designed and the details are still flexible - but if it’s best not to leave that creation to JH when it’s required, I’m sure I can have another element of the system handle it.
That leaves us with just running the server as the right UID/GID, which does indeed seem possible as long as we have access to those values at startup time, and I had indeed seen the thread you reference on getting that information from LDAP. Just to add yet a further complication, if I’m using OpenID Connect to bring in my identities, and the UID and GID are included in the metadata package it returns, am I right in thinking I can apply a similar pre_spawn_start override to grab that data and use it to configure the server startup?
Ultimately Z2JH can only do what Kubernetes allows it to do, so whilst there should be ways to get JupyterHub to change your UIDs/GIDs on external storage (e.g. your suggestion of an init container to chown the subvolume) it’s always going to be easier if that’s handled externally.
The main requirements for passing the UID/GID to the spawner are that you can store the IDs in auth_state in the authenticate() method:
This auth_state is accessible to your spawner if you enable it. Depending on how your authentication works you may need to override authenticate() to request additional metadata and add the extra information to auth_state.
It sounds like you’ve set things up in such as way that your .jupyterhub is created by a root process instead of the user. This directory isn’t created as standard by JupyterHub so it must be due to your configuration.