Jupyterhub-singleuser and jupyter_server

JupyterHub
1

Hi everyone,

I’ve got a question that is more about trying to understand what’s happening instead of asking for support. I have a JupyterHub deployment on K8 using the helm chart (with a significant number of my own modifications), and I’ve been fighting to allow users to delete non-empty directories for a while (it is related to a few long standing github issues, but if you want to go down the rabbit trail you can start here or here). Last year with my annual academic cycle deployment, I ended up just patching the filemanager.py file from the notebook app to allow for deleting non-empty directories, but it is a bit of a kludge to manually patch the file.

This year, when I was creating my single user docker container, I realized that my old patch didn’t work when running Jupyter Lab locally due to JupyterLab 3.0 having switched to jupyter_server as the notebook backend instead of the notebook app. Moreover, I discovered that jupyter_server had two configurables that when set allowed deleting directories (--FileContentsManager.always_delete_dir=True --FileContentsManager.delete_to_trash=False)!

However, when I then set up my JupyterHub k8 deployment with my new single user docker image, those configurables didn’t work, and I had to revert to patching filemanager.py for the notebook app, not jupyter_server.

So, that was the long winded way to get to my real question: Why doesn’t JupyterLab when launched from JupyterHub use the jupyter_server backend and instead uses the notebook app backend? I know that when launching the single user server the “jupyterhub-singleuser” command is ultimately used to launch the server (and the default url needs to be set to “/lab” in order to actually use the Jupyter Lab interface), and it seems that this command makes JupyterLab use the notebook app backend instead of the jupyter_server backend (which is the official backend for JupyterLab 3.*).

So, why does jupyterhub-singleuser work this way? I can find no explanation, and I’m having a really hard time tracking down what exactly that command calls even. I don’t know if this is expected behavior, so I haven’t opened an issue with the JupyterHub repo yet, but if this is not intended behavior, I’m happy to do so. It does seem, since JupyterLab is moving to jupyter_server and (I think) there are plans to sunset the notebook app, at some point JupyterHub will need to also use the jupyter_server backend.

I’d appreciate any insight anyone has on this!

2

The singleuser module in jupyterhub allows the user (developer) to override the Application launched by the single-user image. By default, it is "notebook.notebookapp.NotebookApp"

You can change this by setting the environment variable JUPYTERHUB_SINGLEUSER_APP='jupyter_server.serverapp.ServerApp'

1 Like
3

Ah, gotcha! That’s very helpful. There even looks like the app can be configured using the JUPYTERHUB_SINGLEUSER_APP environmental variable (edit: got ninja’d by an edit). Is there any downside to switching the single user app to jupyter_server?

4

There shouldn’t be any issues for most deployments. There is a shim that copies any NotebookApp configuration to the ServerApp, so if you have Jupyter Notebook installed (via nbclassic), that configuration should still be available.

I’ve been using it in deployment for a while, and no issues.

5

Gotcha. Do you need defaultURL: "/lab" anymore if you switch backends but don’t have nbclassic installed? Will it automatically use JupyterLab?

6

I can’t remember. I’d expect you probably need to set it, because the ServerApp doesn’t prefer any particular URL (as JupyterLab is now a server extension), and I don’t think that JupyterHub sets a default.

7

Just tested things, and you do need the default url set. Thanks for all of your help!

8

FYI JupyterHub will probably switch to jupyter-server/jupyterlab in the 2.0 release: 2.0: jupyterlab by default by minrk · Pull Request #3615 · jupyterhub/jupyterhub · GitHub

I’m surprised you still needed defaultURL: "/lab" though!

9

Yeah, if I didn’t have the “/lab” url, it just said something like “Server is running” when I logged in (until I manually went to “/lab”). I checked and my environment does have nbclassic installed (the image is based on the official Jupyter Docker images), so I can’t say for certain what happens if it’s not installed. However, I don’t think that will have any effect on what is started, though.

Good to know that it’s switching to jupyter-server by default, though.

10

Is there a definitive way to tell if you’re using the legacy notebook app at all? We’re using jupyterhub 1.5.0 and z2jh 1.2.0 with a docker-stacks/scipy-notebook parent image for our singleuser image. We’re calling jupyter-labhub to start the singleuser app. With kubespawner it’s a bit confusing because the container is hard-coded to be named notebook and you see things in the notebook container logs saying notebook (it’s got nbclassic and notebook installed in the image) and we always see this:

Nov 17 09:22:58 jupyter-6192b1ecab4f10c6670c9e5b notebook INFO INFO 2021-11-17T15:22:58.080Z [SingleUserLabApp serverapp:2577] Jupyter Server 1.11.2 is running at:

We just started setting this yesterday in our z2jh hub config:

JUPYTERHUB_SINGLEUSER_APP: “jupyter_server.serverapp.ServerApp”

But that’s currently only in one of our testing clusters and I’m trying to compare the notebook logs and REST API between those environments and it’s hard to see the difference, e.g. GET /user/<userid>/api/ returns the jupyter-server version in both cases, not the notebook app web server version.

I’m thinking the thing that I can use to tell is the absence of this log message:

Nov 11 11:29:24 jupyter-5e18a4193f4a3f001127f809 notebook [I 2021-11-11 17:29:24.785 SingleUserLabApp notebookapp:2302] Jupyter Notebook 6.4.5 is running at:

Is there something better?

11

When you start with jupyter-server (usually via setting JUPYTERHUB_SINGLEUSER_APP or Spawner.cmd = "jupyter-labhub"), you will see the log line:

[I 2021-11-18 09:18:46.022 SingleUserNotebookApp serverapp:2577] Jupyter Server 1.11.2 is running at:

When you start the classic notebook server, it will look like:

[I 2021-11-18 09:19:01.888 SingleUserNotebookApp notebookapp:2329] Jupyter Notebook 6.4.6 is running at:

So both log the same {name of server} {version} running at line identifying which server implementation is used. If you see that “Jupyter Server” log line, it is using Jupyter server and not the classic notebook server.

JupyterHub 2.0 will log the inheritance of the singeluser app more explicitly:

[I 2021-11-08 09:17:37.223 SingleUserLabApp mixins:617] Starting jupyterhub single-user server version 2.0.0rc3
[I 2021-11-08 09:17:37.223 SingleUserLabApp mixins:629] Extending jupyterlab.labhubapp.SingleUserLabApp from jupyterlab 3.2.1
[I 2021-11-08 09:17:37.223 SingleUserLabApp mixins:629] Extending jupyter_server.serverapp.ServerApp from jupyter_server 1.9.0
1 Like
12

We have a wrapper script around the Spawner.cmd but it eventually calls jupyter-labhub. We’re always seeing the Jupyter Server 1.11.2 is running at line so that’s why I was confused about what was being used.

Anyway, I think we’re doing all of the correct things now and I’ll assume that if I don’t see Jupyter Notebook * is running then we’re OK.

This was all originated from my creating a jupyter-server issue and being told I was using notebook:

Which, to be fair, looks like we at least were because of the notebookapp.py in the stacktrace in that issue.

13

In which file should I pass this parameter to?

14

If you’re using JupyterHub>=2.0, you should no longer need to pass this parameter, as jupyter_server is the default server.