KubeSpawner with external NFS and user as non-root

JupyterHub Zero to JupyterHub on Kubernetes
1

Hi all,

First off, the helm chart is great and works like a dream. Great work!
What I want is to use Z2JH using users’ existing home folders over NFS using keycloak against FreeIPA for auth. Because of permissions this means the JH singleuser pod needs to run as the authenticated user. Note, my NFS is not kerberized (yet), so that saves some headache.
I borrowed config and inspiration from here: KubeSpawner and LDAPauthentication run under users LDAP UID - #7 by manics I’ll post the exact details further down.
Currently I need to start the pod as root so it can switch the user. Is there any way to avoid this? I guess not?

Anyway, config time. Maybe this will also help someone else :slight_smile:

singleuser:
  image:
    name: jupyter/datascience-notebook
    tag: latest
  cmd: start-notebook.sh
  uid: 0
  storage:
    type: static
    static:
      pvcName: homes-pvc
    homeMountPath: /home/{username}
hub:
  config:
    Authenticator:
      enable_auth_state: true
    GenericOAuthenticator:
      client_id: secret
      client_secret: secret
      oauth_callback_url: "https://jupyter/hub/oauth_callback"
      token_url: 'https://keycloak/realms/REALM/protocol/openid-connect/token'
      authorize_url: 'https://keycloak/realms/REALM/protocol/openid-connect/auth'
      userdata_url: 'https://keycloak/realms/REALM/protocol/openid-connect/userinfo'
      logout_redirect_url: 'https://keycloak/realms/REALM/protocol/openid-connect/logout'

      login_service: keycloak
      username_key: preferred_username
      userdata_params: 
        state: 'state'
        uid: 'uid'
      tls_verify: false
      claim_groups_key: 'roles'
      admin_groups:
        - admin
      scope:
        - openid
        - profile
        - roles
  extraConfig:
    SpawnerCustomConfig: |
      from oauthenticator.generic import GenericOAuthenticator

      class OauthInfo(GenericOAuthenticator):
          async def pre_spawn_start(self, user, spawner):
              auth_state = await user.get_auth_state()
              self.log.info(f"pre_spawn_start auth_state: {auth_state}")
              if not auth_state: 
                  return
              user_info = auth_state['oauth_user']
              spawner.environment["NB_UID"] = str(user_info["uid"])
              spawner.environment["NB_GID"] = str(user_info["gid"])
              spawner.environment["NB_USER"] = user_info["preferred_username"]
              spawner.environment["GIT_AUTHOR_NAME"] = user_info["name"]
              spawner.environment["GIT_COMMITTER_NAME"] = user_info["name"]
              spawner.environment["GIT_AUTHOR_EMAIL"] = user_info["email"]
              spawner.environment["GIT_COMMITTER_EMAIL"] = user_info["email"]

      c.JupyterHub.authenticator_class = OauthInfo
    CustomHubConfig: |
      c.JupyterHub.cleanup_servers = True

Notes for posterity:

  • I needed to make the NFS PV and PVS beforehand. Use spec.claimRef on the PV.
  • I needed to add LDAP and client mappers to the keycloak client scope
2

I’m not aware of a way to avoid starting as root. The username and uid need to exist inside the container, so although the uid of the pod can be set dynamically you still need root permissions to create a new user or modify the existing one, or you create all users in advance in your image which obviously isn’t very maintainable.

3

To add to what Mike said, I’ll discuss the security implications. You see, in NFSv3 and below, the notion of NFS security is entirely based upon the server trusting the client to not lie. That’s it. The server trusts the client entirely. If you are root on the client, you now have the capability to become any user on the client and also overwrite any file on the server. A compromise of the client would be able to overwrite files owned by root on the server over NFS. Without root squash, root on the client can still impersonate any individual user, but not overwrite files owned by root on the server.

4

Many thanks for your insights.
I guess I was just making sure I didn’t miss an option to inject users before the pod gets spawned.
I’m aware NFS is built on trust (including NFSv4, given you use sec=sys), but it’s good to have that spelled out every so often. The solution is to use kerberos for authentication, but I’m not sure I’m quite ready for that particular can of worms (Get keycloak to forward a TGT, use that to authenticate within the spawned pod?).