JupyterLab server extension not working on JupyterHub

JupyterHub Zero to JupyterHub on Kubernetes
1

Hi there,

I am trying to use a JupyterLab server extension inside JupyterHub. I already checked other threads and I actually just replied to one of them where OP had a very similar issue.

Everything works fine in JupyterLab, and when I deploy the container in the JupyterHub server, I can see that the frontend extensions that I developed work completely fine. But when I try to deploy the server extension, it doesn’t seem to work

To give some context, I am using Zero to JupyterHub with a custom Docker image to install the extensions in the container. Then, inside singleuser in my config.yaml file , I have:

singleuser:
  image:
    name: <repo>/<image>
    tag: <my_tag>

The Dockerfile looks like this:

FROM jupyterhub/singleuser:latest
USER root

RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y git && \
    pip install nbgitpuller

COPY . .

RUN jupyter server extension enable nbgitpuller --sys-prefix && \
    pip install -e . && \
    jupyter server extension enable <my_extension> --sys-prefix

I deploy the jupyterhub server, and everything works fine, until I check the logs, and see that it can’t find the endpoint. When I check the logs for a user that has just spawned a server, one of the first messages I get is:

<my_extension> | error adding extension (enabled: True): The module '<my_extension>' could not be found (No module named 'my_extension>'). Are you sure the extension is installed?

If I go to that pod, I can see that the jupyter labextension list shows <my_extension>, but when checking jupyter server extension list, I get the same issue:

jupyter server extension list
Config dir: /home/jovyan/.jupyter

Config dir: /opt/conda/etc/jupyter
    <my_extension> enabled
    - Validating <my_extension>...
       X The module '<my_extension>' could not be found (No module named '<my_extension>'). Are you sure the extension is installed?
    jupyter_lsp enabled
    - Validating jupyter_lsp...
      jupyter_lsp 2.2.5 OK
...

One thing I noticed is that the files that should be added for backward compatibily are not there even though I added them when I created the extension. This is the setup.py file:

from setuptools import setup

setup(
    name="<my_extension>",
    version="0.1.0",
    packages=["<my_extension>"],
    install_requires=[
        "jupyter_server",
        "tornado"
    ],
    include_package_data=True,
    data_files=[
        ("etc/jupyter/jupyter_server_config.d", ["jupyter-config/server-config/<my_extension>.json"]),
        ("etc/jupyter/jupyter_notebook_config.d", ["jupyter-config/nb-config/<my_extension>.json"])
    ]
)

Can someone point me what I am doing wrong please?

Thanks!

2

Ok, after hours of debugging, I just found the issue.

In case anyone is using a custom Docker image with a server extension (these are my insights, but I might be wrong assuming the way it works):

  • Enabling the extension will add it to the etc/jupyter/jupyter_server_config.d directory, but that doesn’t mean it knows where to find it
  • You need to set a path for finding it, but when you use the Docker container inside singleuser, you need to also bare in mind that your files might get lost if not copied in a directory that you can eventually find.
  • Usually, that path is the same path you used when you installed that extension in your Dockerfile.

So in my case, I copied the values to /tmp, and then copied the content of /tmp in /home/jovyan (the directories can be changed, I actually should change them, so they are hidden to the users)

This is how my current Dockerfile looks:

FROM jupyter/base-notebook:latest
WORKDIR /home/jovyan
USER root

COPY . /tmp

RUN pip install -e /tmp/<my_extension> && \
    jupyter server extension enable <my_extension> --sys-prefix

USER jovyan

And inside config.yaml:

singleuser:
  image:
    name: <my_image>
    tag: <my_tag>
  lifecycleHooks:
    postStart:
      exec:
        command: ["cp", "-r", "../../tmp/.", "/home/jovyan"]

There must be an easier way to deal with this, so if anyone can shed some light on it, it would be great!

3

Short answer, based on some assumptions:

If you don’t need users to edit the extension, removing -e from your pip install is probably the only thing you need to do. Doing a regular pip install ./myextension will copy the extension to the Python site-packages, and the ./myextension directory is not needed and can be removed.

Long answer, including the apparent goal of having files editable by users in $HOME:

You’re exactly right that the problem was that you had an editable install (pip install -e) of a package from a location in $HOME. Editable installs require the original files to remain present in their original location in order to be imported. But since $HOME is a mounted volume, no files in that directory in the image are available in the container (this is generally true for containers, not specific to Jupyter - no image files in the path of a mounted volume are available in a container, the Z2JH part is that $HOME is a volume by default, to enable persistence of user files).

What’s not clear from the description is why your extension needs to be editable and in $HOME. The simplest fix for making the extension available is like you have done, to install from /tmp or remove -e from your pip install command. There is no particular reason to use pip install -e here, as a regular pip install will suffice and avoid any reliance on files in $HOME (or wherever you install from).

However, given that you’ve added cp -r to copy files (no longer the installed version of the package) into $HOME at startup, I assume you want the files to be visible/editable by the user. The cp -r approach means:

  1. $HOME will contain a copy of the files from the image
  2. files will be editable by the user,
  3. but user edits will not affect the installed package (even with -e, because sys.path still points to the /tmp/my_extension copy)
  4. on every server start, the edits by the user will be lost as cp -r clobbers the files in $HOME with those from the image.

If you can give some more information about the use case why/how you want users to be able to edit the extension, we might be able to come up with something.

2 Likes
4

Oh my god, thank you so much @minrk. I simply removed the -e, removed the cp -r part and it worked, I can’t believe I banged my head against the wall for 2 days for this…

I first added the editable part because it was needed for the project, but then we didn’t need users to edit the package anymore. The initial idea was that the extensions could be changed upon spawning the server, but then we realized we didn’t actually need it anymore, and then I forgot to remove that flag. Tbh, I didn’t know that behaviour of the editable packages, but now seeing it in perspective, it makes a lot of sense.

Thank you so much for your help! You’re a hero :blush:

1 Like