JupyterHub 5.1 failure to create PVC when email used as username_claim

JupyterHub Zero to JupyterHub on Kubernetes
, ,
1

Hi,

I have JH 4.1.5 (using the helm chart v3.3.7) deployed in IBM Cloud OpenShift using keycloak as the authenticator and email as the username_claim. This is all working nicely and it creates private notebook PVCs of the format 'claim-{sanitised username} where {sanitised-username} has the @ and . symbols replaced with -40 and -2e respectively.

But… with the development version of JH 5.1 it does not sanitise the username when building the PVC name which consequently fails in ROKS as the name must a-zA-z. The error is:

"kind":"PersistentVolumeClaim","causes":[{"reason":"FieldValueInvalid","message":"Invalid value: \"xxxxxx@bbbb.com\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character

How do I get around this? I need to continue to use email as the username_claim and the PVC name generated has to match that used by JH 4.1.5 so that I can upgrade and users will not loose all their data.

I have read some commentary about custom authenticators to sanitise the username but this will mean the username in the top right will be changed, plus the user prompt in a terminal etc. I am trying to minimize the impact to users.

I also understand this is the dev version and not a stable version. Could this be on the list for fixing for the stable version?

Thanks,
Ian

2

Is this kubespawner 7 beta or 6? This is a kubespawner matter i think

3

I am using the helm chart version 4.0.0-0.dev.git.6760.h7f442465 from the website. I have narrowed the issue some more. Its to do with the metadata.label being set to my username (aka email address), not the claim name as I said previously.

"kind":"PersistentVolumeClaim","causes":[{"reason":"FieldValueInvalid","message":"Invalid value: \"xxxxx@yyyy.com\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')","field":"metadata.labels"}]}

If I use surname for ex as the username_claim, then the metadata.label is set to the surname. Anyone know where I can overwrite this label?

I have added a pre-spawner hook to correct the PVC name to be consistent with JH 4.1.5 but now trying to work out where that label is coming from.

  extraConfig:
    pvc-name: |
      from kubespawner import KubeSpawner

      class CustomSpawner(KubeSpawner):
        async def pre_spawn_hook(self, dummy):
          new_pvc_name = self.user.name
          sanitized_pvc_name = new_pvc_name.replace('@', '-40').replace('.', '-2e')
          self.pvc_name = f'claim-{sanitized_pvc_name}'
          self.log.warning(f'>>> new: {self.pvc_name}')

      c.JupyterHub.spawner_class = CustomSpawner

In JH 4.1.5 the metadata label is set to the sanitised username. This is not happening in JH 5.1.

4

Im on mobile i cant check, but i think you are now using kubespawner beta1 with changes related to pvc naming

However, they were designed to be non breaking by remembering previous names etc, but by actively setting pvc_name the mitigation of the breaking change may have failed or been bypassed.

5

The point of the changes were that you wouldnt need something like this to begin with.

Kubespawners changelog should be linking to some docs of relevance to read

6

Thanks @consideRatio.

Issue resolved by making the following changes:

hub:  
  extraConfig:
    jupyterhub_config.py: |
      c.KubeSpawner.slug_scheme = "escape"
      c.KubeSpawner.storage_extra_labels = {'hub.jupyter.org/username': '{escaped_username}'}
      c.KubeSpawner.extra_labels = {'hub.jupyter.org/username': '{escaped_username}'}

singleuser:
  storage:
    static:
      subPath: "{escaped_username}"
    dynamic:
      pvcNameTemplate: claim-{escaped_username}{servername}
      volumeNameTemplate: volume-{escaped_username}{servername}

This was the only thing I could do to correct the metadata label issue and have JH 5.1 look & feel like JH 4.1.5.

Many thanks.

7

Thanks for testing!

Indeed, this is exactly the kind of case that is meant to require no configuration, but some bugs in the new slug scheme prevented emails from being escaped properly. This should be fixed by fix some safe slug patterns by minrk · Pull Request #856 · jupyterhub/kubespawner · GitHub

1 Like