Get username with dockerspawner

JupyterHub
1

Hello,

I’m installing jupyterhub in a swarm cluster, with LTI authentication via Moodle
For this, I use dockerspawner.

Containers must use homes from another platform, available from an nfs server. To do this, I want to start the individual contents using extra_container_spec, then providing a UID that I calculate based on the ID provided by moodle (which corresponds to spawner.user.name).
For example, for a moodle ID equal to 1, I add 1000 to obtain a UID of 1001. the container then starts with ID 1001 which corresponds to the owner of the desired home.
So, in jupyterhub_config.py, the extra_container_spec method is described like this:

c.JupyterHub.spawner_class = 'dockerspawner.SwarmSpawner'
.
.
.
c.SwarmSpawner.extra_container_spec = {
    'user': user_ID,
}

The problem is that I can’t get the moodle ID. I’m desperately trying to use the get_env function in the following way:

my_env=c.Spawner.get_env

…But I can’t extract the username variable.
Could anyone help me?

thanks in advance

1 Like
2

You can inject the user parameter directly into the spawner environment which will be eventually merged with extra_container-spec. You can make c.Spawner.environment a callable to get spawner.user.name which corresponds to your moodle ID. Check this topic on how to do it.

1 Like
3

Thank you for your answer…
I tried to use this example, but i had no success… Here is what i have done:

def set_custom_UID(spawner):
  custom_UID=int(spawner.user.name) + 1000
  return str(custom_UID)

c.Spawner.environment = { 
    'USER_ID' : set_custom_UID, 
}

To debug, I tried to simplify by forcing a static uid, just to see if that already worked:

c.Spawner.environment = { 
    'USER_ID' : '1001', 
}

…But it doesn’t work…
If I understood correctly, there is no need to force a new UID when calling the extra_container_spec method… right?
So, there is no more need of these lines:

c.SwarmSpawner.extra_container_spec = {
    'user': <my id that i can not get in jypyterhub_config.py>,
}
4

I have looked into it too quickly. Well, the solution of injecting user ID into spawner environment wont work as user arg in ContainerSpec that needs to be passed using extra_container_spec. Sorry about wrong lead.

Seems like dockerSpawner have extra_create_kwargs that takes a callable as input but SwarmSpawner does not expose such a config parameter. Well, in this case you can create a custom spawner subclassing from SwarmSpawner something like below in your jupyterhub_config.py

from dockerspawner import SwarmSpawner

class MySwarmSpawner(SwarmSpawner):
    async def create_object(self):
        uid = int(self.user.name) + 1000
        self.extra_container_spec.update({"user": str(uid)})
        return super().create_object()

c.JupyterHub.spawner_class = 'MySwarmSpawner'
1 Like
5

I replaced th line

c.JupyterHub.spawner_class = 'dockerspawner.SwarmSpawner'

with your code, but the hub failed to start with the logs:

 | [C 2024-04-04 12:53:03.475 JupyterHub application:115] Bad config encountered during initialization: The 'spawner_class' trait of <jupyterhub.app.JupyterHub object at 0x7f118146a410> instance must be a type, but 'MySwarmSpawner' could not be imported
6

Try removing quotes around MySwarmSpawner eg c.JupyterHub.spawner_class = MySwarmSpawner

7

Hello,

Would you share your full configuration of jupyterhub_config.py
Because i am also using swarmspawner, but did´t make it.

Thanks,

8

Here is my original jupyterhub_config.py (before overloading the creator of SwarmSpawner):

import os
import shutil
import sys
import subprocess
from ltiauthenticator import LTIAuthenticator
from subprocess import check_call

# The proxy is in another container
c.ConfigurableHTTPProxy.should_start = False
c.ConfigurableHTTPProxy.api_url = 'http://proxy:8001'

c.JupyterHub.spawner_class = 'dockerspawner.SwarmSpawner'

c.JupyterHub.shutdown_on_logout = True
c.JupyterHub.init_spawners_timeout = 60

c.SystemUserSpawner.name_template = '{prefix}-{username}-{servername}'
c.JupyterHub.allow_named_servers = True

def create_dir_hook(spawner):
    """Create directory"""   
    moodlename = spawner.user.name  # get the username retrurned after lti auth (it's an integer)
    username = 'jupyter-' + moodlename  ## i users and home are formatted like jupyter-<moodlename>
    volume_path = os.path.join('/volume/home', username)
    user_rights=username + ':users'
    user_id = int(moodlename) + 1000
    converted_user_id = str(user_id)
    #os.system("useradd -u "+ converted_user_id + " " + username) ## maybe nor necessray for the moment
    if not os.path.exists(volume_path):
        os.mkdir(volume_path, 0o755)
        src_path = '/tmp/jovyan/'
        shutil.copytree(src_path, volume_path, dirs_exist_ok=True)
        subprocess.run(["chown", "-R", user_rights, volume_path])
        subprocess.run(["chmod", "-R", "+w", volume_path])        

def clean_dir_hook(spawner):
    """Delete directory"""
    username = spawner.user.name  # get the username
    temp_path = os.path.join('/volume/home', username, 'temp')
    if os.path.exists(temp_path) and os.path.isdir(temp_path):
        shutil.rmtree(temp_path)

c.Spawner.pre_spawn_hook = create_dir_hook
c.Spawner.post_stop_hook = clean_dir_hook

c.DockerSpawner.volumes = { '/data/docker/test_jupyter/swarm_stack_1/test-perso_ssl/volume/home/jupyter-{username}' : '/home/jovyan' }

c.DockerSpawner.image_whitelist = {
    "latest" : "jupyterhub/singleuser",
    "R and Spark Stack" : "quay.io/jupyter/all-spark-notebook",
    "Tensor Flow" : "quay.io/jupyter/tensorflow-notebook",
    "Julia" : "quay.io/jupyter/julia-notebook",
    "Data Science" : "127.0.0.1:5000/datascience-notebook:02.04.24",
}

c.JupyterHub.hub_ip = '0.0.0.0'
c.JupyterHub.hub_connect_ip = 'hub'
c.SwarmSpawner.network_name = 'jupyterhub-net'
c.SwarmSpawner.extra_host_config = {'network_mode': 'jupyterhub-net'}
c.SwarmSpawner.spawn_timeout = 600

#### at this point, I would like to have a variable containing the moodle username (integer) added to 1000..
### this works fine if I simply replace <forced_uid> with '1001' (if a want the user jovyan to access the home of jupyter-1 wich has UID 1001)

c.SwarmSpawner.extra_container_spec = {
    'user': <forced_uid>,
}

c.Spawner.cmd=["jupyter-labhub"]


### authentication lti for moodle:
c.JupyterHub.authenticator_class = "ltiauthenticator.lti11.auth.LTI11Authenticator"
c.LTI11Authenticator.consumers = {
   "xxxxx":"xxxxxx"
}
1 Like
9

I have removed the quotes…
Now, the hub starts correctly, but the individual container failed to spawn with the message:

Error: HTTP 500: Internal Server Error (Unhandled error starting server 21)

(i tried with the user 21)

here is my new config file:

import os
import shutil
import sys
import subprocess
from ltiauthenticator import LTIAuthenticator
from subprocess import check_call

# The proxy is in another container
c.ConfigurableHTTPProxy.should_start = False
c.ConfigurableHTTPProxy.api_url = 'http://proxy:8001'

#c.JupyterHub.spawner_class = 'dockerspawner.SwarmSpawner'

from dockerspawner import SwarmSpawner

class MySwarmSpawner(SwarmSpawner):
    async def create_object(self):
        uid = int(self.user.name) + 1000
        self.extra_container_spec.update({"user": str(uid)})
        return super().create_object()
        
c.JupyterHub.spawner_class = MySwarmSpawner

c.JupyterHub.shutdown_on_logout = True
c.JupyterHub.init_spawners_timeout = 60
c.SystemUserSpawner.name_template = '{prefix}-{username}-{servername}'
c.JupyterHub.allow_named_servers = True
c.JupyterHub.log_level = 10

def create_dir_hook(spawner):
    """Create directory"""   
    moodlename = spawner.user.name  # get the username
    username = 'jupyter-' + moodlename
    volume_path = os.path.join('/volume/home', username)
    user_rights=username + ':users'
    user_id = int(moodlename) + 1000
    converted_user_id = str(user_id)
    os.system("useradd -u "+ converted_user_id + " " + username)
    if not os.path.exists(volume_path):
        os.mkdir(volume_path, 0o755)
        src_path = '/tmp/jovyan/'
        shutil.copytree(src_path, volume_path, dirs_exist_ok=True)
        subprocess.run(["chown", "-R", user_rights, volume_path])
        subprocess.run(["chmod", "-R", "+w", volume_path])        

def clean_dir_hook(spawner):
    """Delete directory"""
    username = spawner.user.name  # get the username
    temp_path = os.path.join('/volume/home', username, 'temp')
    if os.path.exists(temp_path) and os.path.isdir(temp_path):
        shutil.rmtree(temp_path)

c.Spawner.pre_spawn_hook = create_dir_hook
c.Spawner.post_stop_hook = clean_dir_hook

c.DockerSpawner.volumes = { '/data/docker/test_jupyter/swarm_stack_1/test-perso_ssl/volume/home/jupyter-{username}' : '/home/jovyan' }

c.DockerSpawner.image_whitelist = {
    "latest" : "jupyterhub/singleuser",
    "R et Spark Stack" : "quay.io/jupyter/all-spark-notebook",
    "Tensor Flow" : "quay.io/jupyter/tensorflow-notebook",
    "Julia" : "quay.io/jupyter/julia-notebook",
   "Data Science" : "127.0.0.1:5000/datascience-notebook:02.04.24",
}

c.JupyterHub.hub_ip = '0.0.0.0'
c.JupyterHub.hub_connect_ip = 'hub'
c.SwarmSpawner.network_name = 'jupyterhub-net'
c.SwarmSpawner.extra_host_config = {'network_mode': 'jupyterhub-net'}
c.SwarmSpawner.spawn_timeout = 600

c.Spawner.cmd=["jupyter-labhub"]

# debug-logging for testing
import logging
c.JupyterHub.log_level = logging.DEBUG

c.JupyterHub.authenticator_class = "ltiauthenticator.lti11.auth.LTI11Authenticator"
c.LTI11Authenticator.consumers = {
   "xxxx":"xxxx"
}

here are the logs of the hub:

Unhandled error starting 21's server: 'coroutine' object is not subscriptable
	Traceback (most recent call last):
       File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 798, in spawn
         url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
       File "/usr/local/lib/python3.10/dist-packages/dockerspawner/dockerspawner.py", line 1310, in start
         self.object_id = obj[self.object_id_key]
     TypeError: 'coroutine' object is not subscriptable
1 Like
10

Ahh, I think it is due to async directive on create_object. Could you try removing async so it should be

class MySwarmSpawner(SwarmSpawner):
    def create_object(self):
        uid = int(self.user.name) + 1000
        self.extra_container_spec.update({"user": str(uid)})
        return super().create_object()

As we are returning a async method in our overloaded method, we dont need to use async for overloaded method.

1 Like
11

Great !!! it works !!!
Thank you very much !

Now, my containers start with the good linux uid and access the desired home folder.
The goal was to permit the users of my new jupyterhub/swarm to access the common home of the old jupyterhub/tljh (the swarm version will be tested during some months).
So, i will surely come back soon to customise tljh for fixing the uid too and ensure that any moodle user will have the same uid on each platform.

thanks again!

So, to recap, here is the code to put in jupyterhub_config.py:

from dockerspawner import SwarmSpawner

class MySwarmSpawner(SwarmSpawner):
    def create_object(self):
        uid = int(self.user.name) + 1000
        self.extra_container_spec.update({"user": str(uid)})
        return super().create_object()
        
c.JupyterHub.spawner_class = MySwarmSpawner
2 Likes