Mount external NFS ( or local filesystem )

JupyterHub Zero to JupyterHub on Kubernetes
#1

Hi,
We have manage to configure z2jh to spin up containers ran by specific UID/GID from our LDAP, but now we need those containers to be able to mount an NFS mount from outside of k8s, or we can also mount that NFS on each host from the k8s cluster and then mount local paths into the containers. Is that possible with z2jh ?

Cheers.

2020: The Year in Review
#2

If you can make your external mounts appear as a Kubernetes PVC you can mount them into your user containers with Z2JH:

#3

Hi @manics,
How would I do that? Is it something I can do from k8s? Do you have any docu?

Thanks.

#4

Yes, you could create it manually in K8S, then add it to your Z2JH config.

#5

Hi @manics have created PV and PVC and it works fine, but when I try to mount it under /home, the spawn of the singleuser container fails due to impossibility of creation of /home/jovyan

2020-09-28 13:56:58+00:00 [Warning] Error: failed to start container "notebook": Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"mkdir /home/jovyan: permission denied\"": unknown

I am not sure at which point it is trying to create default user’s home, the helm chart looks like:

  extraEnv:
    CHOWN_HOME: 'yes'
  uid: 0
  fsGid: 0
  serviceAccountName:
  storage:
    type: static
    extraLabels: {}
    static:
      pvcName: pvc-nfs-home
      subPath: 'home/{username}'
    capacity: 10Gi
    homeMountPath: /home
#6

Is your Dockerfile publicly available?

#7

that is basically my Dockerfile

20:16 # cat /root/test-nb/datascience-notebook/Dockerfile 
FROM jupyter/datascience-notebook:latest
USER root
ADD ./passwd /etc/passwd
ADD ./group /etc/group
USER $NB_UID
#8

it was a typo…sorry. Now I can mount it in /home/{username} and $HOME is properly set up, but still /home/jovyan is what you see when you open a terminal. the if you do :
cd $HOME

you get to your home.
image

#9

Kubespawner has a working_dir config option

But this isn’t exposed in Z2JH. You should however be able to set it using hub.extraConfig. Try something like

hub:
  extraConfig: |
    c.KubeSpawner.working_dir = '/home/{username}'
#10

It seems that if I set it to:

 c.KubeSpawner.working_dir = '/home'

it works, but when adding {username} I get :

Error: failed to start container "notebook": Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "chdir to cwd (\"/home/moscardo\") set in config.json failed: permission denied": unknown

Is it maybe because it is trying to set the working directory as root and because NFS is no_root_squash it needs to be the user? Or maybe because the share is still not mounted at this point?

#11

The share should be mounted before the pod starts. It’s possible your NFS server restricts the chown operation or imposes some other restrictions. Can you try running chmod on the NFS server to change the UID of the directory to your user, and remove CHOWN_HOME: 'yes' from your Z2JH config?

#12

Root cannot make the change, since it is mounted with no_root_squash option, and user=moscardo can do it:

[root@login home] chown 23446:100 /home/moscardo
chown: changing ownership of ‘moscardo’: Operation not permitted

[moscardo@login ~]$ chown 23446:100 /home/moscardo
[moscardo@login ~]$

However, all the home directories in that share have proper permissions that there is no need for changing them.

Update: It seems that doesn’t matter what I set to CHOWN_HOME that it always tries to change it

#13

@manics How do you do this with a custom spawner?

class CustomSpawner(KubeSpawner):
     def get_env(self):
         env = super().get_env()
         env.update({
             'USER_INFO': get_user_info()
         })
         return env


c.JupyterHub.spawner_class = CustomSpawner