Starting Server for Non-Default Users in JupyterHub: 500 Internal Server Error

JupyterHub
,
1

I am encountering an issue with starting a server for non-default users in JupyterHub.

image
image1170×524 38 KB

When attempting to start a server for a user named “mahdi” (or any other non-default user), I receive the following error message in the JupyterHub container logs:

[I 2023-09-20 07:40:15.461 JupyterHub provider:659] Creating oauth client jupyterhub-user-mahdi
[I 2023-09-20 07:40:15.474 JupyterHub spawner:1689] Spawning jupyterhub-singleuser
[E 2023-09-20 07:40:15.479 JupyterHub user:884] Unhandled error starting mahdi's server: Exception occurred in preexec_fn.
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.10/site-packages/jupyterhub/user.py", line 798, in spawn
        url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
      File "/opt/conda/lib/python3.10/site-packages/jupyterhub/spawner.py", line 1699, in start
        self.proc = Popen(cmd, **popen_kwargs)
      File "/opt/conda/lib/python3.10/subprocess.py", line 971, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/opt/conda/lib/python3.10/subprocess.py", line 1864, in _execute_child
        raise child_exception_type(err_msg)
    subprocess.SubprocessError: Exception occurred in preexec_fn.
    
[E 2023-09-20 07:40:15.492 JupyterHub pages:313] Error starting server mahdi: Exception occurred in preexec_fn.
    Traceback (most recent call last):
    NoneType: None
    
[W 2023-09-20 07:40:15.492 JupyterHub web:1852] 500 GET /hub/spawn/mahdi (::ffff:172.17.0.1): Unhandled error starting server mahdi
[E 2023-09-20 07:40:15.493 JupyterHub log:183] {
      "X-Forwarded-Host": "127.0.0.1:8000",
      "X-Forwarded-Proto": "http",
      "X-Forwarded-Port": "8000",
      "X-Forwarded-For": "::ffff:172.17.0.1",
      "Sec-Fetch-User": "?1",
      "Sec-Fetch-Site": "same-origin",
      "Sec-Fetch-Mode": "navigate",
      "Sec-Fetch-Dest": "document",
      "Upgrade-Insecure-Requests": "1",
      "Cookie": "jupyterhub-hub-login=[secret]; _xsrf=[secret]; username-127-0-0-1-8888=[secret]; oauthenticator-state=[secret]; jupyterhub-session-id=[secret]",
      "Connection": "close",
      "Referer": "http://127.0.0.1:8000/hub/admin",
      "Accept-Encoding": "gzip, deflate, br",
      "Accept-Language": "en-US,en;q=0.5",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
      "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0",
      "Host": "127.0.0.1:8000"
    }
[E 2023-09-20 07:40:15.493 JupyterHub log:191] 500 GET /hub/spawn/mahdi (jovyan@::ffff:172.17.0.1) 60.88ms

Furthermore, when I access the URL http://127.0.0.1:8000/hub/spawn/mahdi, I receive a “500 Internal Server Error” with the message “Unhandled error starting server mahdi. You can try restarting your server from the home page.”

It is worth noting that I am able to start a notebook server successfully for the default user “jovyan” (${NB_USER}), but I encounter this issue when attempting to start a server for users such as “mahdi”, “mehran”, or “dainal”.

Here is the Dockerfile I am using for reference:

# Use an official JupyterLab 3.6.3 image as parent image
FROM jupyter/scipy-notebook:lab-3.6.3

# Copy requirements.txt and install Python dependencies
COPY --chown=${NB_UID}:${NB_GID} requirements.txt /tmp/
RUN pip install --quiet --no-cache-dir --requirement /tmp/requirements.txt && \
    fix-permissions "${CONDA_DIR}" && \
    fix-permissions "/home/${NB_USER}"

# Install JupyterLab extensions
RUN pip install jupyterlab-git==0.42.0 && \
    pip install jupyterlab_sql

# Install passlib
RUN pip install passlib

# Install firstuseauthenticator
RUN pip install jupyterhub-firstuseauthenticator

# Copy your .env file into the working directory
COPY --chown=${NB_UID}:${NB_GID} env /home/${NB_USER}

# Load environment variables and clone your Git repository
RUN export $(grep -v '^#' env | xargs) && \
    git clone https://${GIT_USER}:${GIT_TOKEN}@gitlab.scialb.fr/inria/organization/data/jupyterlab-reports.git

# Generate a new JupyterHub configuration file and set default URL to '/lab'
RUN jupyterhub --generate-config && \
    sed -i "s/#c.Spawner.default_url = ''/c.Spawner.default_url = '\/lab'/" jupyterhub_config.py

# Define users and their roles
RUN echo "users = {'allowed': ['mehran', 'danial'], 'admin': ['mahdi', 'jovyan']}" >> jupyterhub_config.py  # Define the users and their roles

# Set the JupyterHub admin user to mahdi and set a hashed password
RUN echo "from passlib.hash import sha256_crypt; print(sha256_crypt.hash('jupyter'))" | python3 > hashed_password.txt && \
    echo "c.Authenticator.admin_users = set(users['admin'])" >> jupyterhub_config.py  # Set the admin users

# Set the authenticator class and create users setting
RUN echo "c.JupyterHub.authenticator_class = 'firstuseauthenticator.FirstUseAuthenticator'" >> jupyterhub_config.py && \
    echo "c.FirstUseAuthenticator.create_users = True" >> jupyterhub_config.py  # Set the authenticator class and allow new users to create an account on their first login

# Set the whitelist, blacklist, and open signup settings
RUN echo "c.Authenticator.whitelist = set(users['allowed'])" >> jupyterhub_config.py  # Set the whitelist of users who are allowed to log in
RUN echo "c.Authenticator.open_signup = False" >> jupyterhub_config.py  # Disable open signup, meaning that new users cannot create an account without admin approval

# Switch to root user
USER root

# Add system users
RUN adduser --disabled-password --gecos "" mahdi && \
    adduser --disabled-password --gecos "" danial && \
    adduser --disabled-password --gecos "" mehran
    
# Change ownership and permissions of home directories
RUN chown -R mahdi:mahdi /home/mahdi && chmod -R 755 /home/mahdi
RUN chown -R mehran:mehran /home/mehran && chmod -R 755 /home/mehran
RUN chown -R danial:danial /home/danial && chmod -R 755 /home/danial

# Add users to 'users' group
RUN usermod -a -G users mahdi
RUN usermod -a -G users mehran
RUN usermod -a -G users danial

# Switch back to non-root user 
USER ${NB_USER}

Note that each user in JupyterHub has a corresponding system user account on the machine.

(base) jovyan@203736ba2dda:~$ awk -F'[/:]' '{if ($3 >= 1000 && $3 != 65534) print $1}' /etc/passwd
jovyan
mahdi
danial
mehran
(base) jovyan@203736ba2dda:~$ id mahdi
uid=1001(mahdi) gid=1000(mahdi) groups=1000(mahdi),100(users)
(base) jovyan@203736ba2dda:~$ cd ..
(base) jovyan@203736ba2dda:/home$ ls
danial  jovyan  mahdi  mehran
(base) jovyan@203736ba2dda:/home$
2

I assume you are running JupyterHub as joyvan user and hence, you can spawn notebooks as that user without any issues. But to spawn as other users, joyvan should have privileges to set ownership of the notebook process to that particular user.

Could you try running JupyterHub with sudo to check if the problem persists?

1 Like
3

Merci for your reply. Yes, I granted sudo privileges to the user running the JupyterHub server, and now everything is working well.

docker run -p 8000:8000 -e GRANT_SUDO=yes --user root --name SCIHUB --rm lab1 jupyterhub