Websocket http error 403

Hi,
I have a websocket error while trying to use the singleuser notebook.
But if I reload the page the error is gone and everything is working like a charm.

For the context I am using an application (which I do not develop) which connect to jupyterhub using a client secret. I am also using a custom jupyterhub authenticator which will make call on the other application to authenticate users.

Everything is working fine, the only problem I encounter is that my application is providing me a link to jupyterhub to access a singleuser notebook with a url like:
/user/myuser/?token=xxx

This link is working and I can access the notebook, but it seems not to allow the websocket connection.
I get 403 errors forbidden on client side and I got this on server side:

[W 2023-08-29 11:04:49.673 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-08-29 11:04:49.674 ServerApp] 403 GET /user/admin/api/kernels/6afe5257-5d57-4ecd-9841-904c15a1f227/channels?session_id=476e41e5-25b8-4fa5-81ec-90ecb50d8baf (@10.40.124.1) 1.56ms
[I 2023-08-29 11:04:49.699 ServerApp] 200 GET /user/admin/api/kernels/6afe5257-5d57-4ecd-9841-904c15a1f227?1693307089686 (admin@10.40.124.1) 0.99ms

But I reload the page and login with the credentials of my other application everything is working fine.
Is there a different authentication methods for the websocket or something else as it seems that everything except the websocket is fine by passing the token in the url.

Thank you

Can you provide more context here? It sounds like the application providing that token may not be providing the right token.

Hi,
More context about which part?
And I am pretty sure that the token is valid otherwise I should not be able to access the interface, see / modify the files.
Maybe there are 2 types of authentication one for the default http calls and one for the wss, but I have no clue on this.

More context about which part?

Sorry, the application creating this link. Creating a link with credentials is a hairy business, and might not be done quite right.

Unfortunately, “a valid token” isn’t sufficiently specific. Each token has an owner and specific permissions associated with it, so a valid token may be permitted to take some actions but not others. That’s why I meant to ask for more context about the application that is acquiring this token and producing this link.

Websocket authentication is handled differently from other API requests, because websockets cannot set the Authorization header (in a browser). As a result, they rely on cookie authentication. So it’s possible the sequence of requests is not resulting in successfully setting the cookie, and your “other application” does. But there’s not enough info here yet to tell what the difference is between the two applications involved.

My guess is that the token-authenticated request for the /lab page is not setting the cookie when it should (there are some checks to prevent setting unnecessary cookies for token-authenticated requests, but I’m guessing this is being applied one too many places). If this is the case, I would guess that when you see this websocket error, editing the URL to remove ?token... and refreshing the page should result in a 403 or login redirect for the whole page. That would mean the cookie isn’t being set. If that’s the case, a more complete log of all the requests to the single-user server would help track down where it’s getting missed.

Can you share your versions of jupyter-server, jupyterlab, and jupyterhub? The startup logs of the single-user session should have most of this information (versions, extensions, etc.) which should be relevant.

Thank you a lot for all these informations !
The application that I am trying to connect is Xnat (which as a jupyterhub plugin to handle this stuff https://wiki.xnat.org/jupyter-integration).

I did not developped this plugin but I had to look into as it is designed for docker (with the dockerspawner) and had to customize it (which seems to work atm).

Here is the code of the custom jpuyterhub authenticator for xnat:

# Create service account for XNAT
c.JupyterHub.services = [
    {
        "name": "xnat",
        "api_token": "secret-token"
    },
]

c.JupyterHub.load_roles = [
    {
        "name": "admin-hub-role",
        "scopes": [
            "admin:servers", "admin:users", "read:hub", "tokens"
        ],
        "services": [
            "xnat"
        ],
    }
]
# Authentication config
class XnatAuthenticator(Authenticator):
    """
    Used to authenticate a user with XNAT

    Requires the environmental variable JH_XNAT_URL to contain the URL of the XNAT to authenticate the user with.
    """
    async def authenticate(self, handler, data):
        xnat_url = f'{os.environ["JH_XNAT_URL"]}'
        xnat_auth_api = f'{xnat_url}/data/services/auth'
        # return {'name': data['username']}
        logger.debug(f'User {data["username"]} is attempting to login.')

        response = requests.put(
            xnat_auth_api, data=f'username={data["username"]}&password={data["password"]}')

        if response.status_code == 200:
            logger.info(f'User {data["username"]} authenticated with XNAT.')
            return {'name': data['username']}
        else:
            logger.info(
                f'Failed to authenticate user {data["username"]} with XNAT.')
            return None

c.JupyterHub.authenticator_class = XnatAuthenticator

So here is what I understand of the authentication process (you can take a look here at the plugin):

Xnat and jupyterhub share a secret to authenticate each other, user from xnat inside jupyterhub when they ask to create a notebook.
When the notebook is up (imo, there is no issue before), there is a button on Xnat which permit to open the link to the notebook.

XNAT.plugin.jupyterhub.servers.goTo = function(server_url) {
        XNAT.plugin.jupyterhub.users.tokens.create().then(token => {
            console.log(token)
            window.open(`${server_url}?token=${token['token']}`, '_blank');
        }).catch(() => {
            window.open(server_url, '_blank');
        })
    }

So it call the backend for the creation of the token, and then create a new window with the url to the lab + the token at the end.

private String tokenUrl(final String username, final String tokenId) {
         return jupyterHubApiUrl + "/users/" + username + "/tokens";
    }

So I think the issue come from the windows that is opened which does not have any cookie sets.
If you have any idea how to pass a properly set up cookie to a new page.

For the version I am using the latest chart of jupyterhub:
jupyterhub/jupyterhub 3.0.2 4.0.2
With the latest image:
name: jupyterhub/k8s-hub tag: “3.0.2”
name: jupyterhub/configurable-http-proxy tag: “4.5.6”
name: traefik tag: “v2.10.4”
name: jupyterhub/k8s-secret-sync tag: “3.0.2”
name: jupyterhub/k8s-network-tools tag: “3.0.2”
name: jupyterhub/k8s-singleuser-sample tag: “3.0.2”
name: registry.k8s.io/kube-scheduler tag: “v1.26.7”
name: registry.k8s.io/pause tag: “3.9”
name: jupyterhub/k8s-image-awaiter tag: “3.0.2”

Can you share the logs of the single-user server (including startup with extensions, versions, etc.)?

Sure, here they are:

#Starting
[I 2023-09-04 11:30:48.450 ServerApp] Package jupyterhub took 0.0000s to import
[I 2023-09-04 11:30:48.463 ServerApp] Package jupyter_lsp took 0.0133s to import
[W 2023-09-04 11:30:48.464 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-09-04 11:30:48.472 ServerApp] Package jupyter_server_terminals took 0.0076s to import
[I 2023-09-04 11:30:48.551 ServerApp] Package jupyterlab took 0.0787s to import
[I 2023-09-04 11:30:48.590 ServerApp] Package nbclassic took 0.0014s to import
[W 2023-09-04 11:30:48.591 ServerApp] A `_jupyter_server_extension_points` function was not found in nbclassic. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-09-04 11:30:48.592 ServerApp] Package nbgitpuller took 0.0008s to import
[I 2023-09-04 11:30:48.593 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-09-04 11:30:48.593 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-09-04 11:30:48.593 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2023-09-04 11:30:48.628 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-09-04 11:30:48.629 JupyterHubSingleUser] Starting jupyterhub single-user server extension version 4.0.2
[I 2023-09-04 11:30:48.629 JupyterHubSingleUser] Using default url from server extension lab: /lab
[I 2023-09-04 11:30:48.632 ServerApp] jupyterhub | extension was successfully linked.
[W 2023-09-04 11:30:48.645 LabApp] 'extra_template_paths' was found in both NotebookApp and ServerApp. This is likely a recent change. This config will only be set in NotebookApp. Please check if you should also config these traits in ServerApp for your purpose.
[I 2023-09-04 11:30:48.648 ServerApp] jupyterlab | extension was successfully linked.
[W 2023-09-04 11:30:48.671 NotebookApp] 'extra_template_paths' was found in both NotebookApp and ServerApp. This is likely a recent change. This config will only be set in NotebookApp. Please check if you should also config these traits in ServerApp for your purpose.
[I 2023-09-04 11:30:48.673 ServerApp] nbclassic | extension was successfully linked.
[I 2023-09-04 11:30:48.673 ServerApp] nbgitpuller | extension was successfully linked.
[I 2023-09-04 11:30:48.954 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-09-04 11:30:48.971 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-09-04 11:30:48.976 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-09-04 11:30:48.977 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-09-04 11:30:48.983 JupyterHubSingleUser] Updating Hub with activity every 300 seconds
[I 2023-09-04 11:30:48.984 ServerApp] jupyterhub | extension was successfully loaded.
[I 2023-09-04 11:30:48.990 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.11/site-packages/jupyterlab
[I 2023-09-04 11:30:48.990 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
[I 2023-09-04 11:30:48.991 LabApp] Extension Manager is 'pypi'.
[I 2023-09-04 11:30:48.993 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-09-04 11:30:48.996 ServerApp] nbclassic | extension was successfully loaded.
[I 2023-09-04 11:30:48.997 ServerApp] nbgitpuller | extension was successfully loaded.
[I 2023-09-04 11:30:48.997 ServerApp] Serving notebooks from local directory: /home/jovyan
[I 2023-09-04 11:30:48.997 ServerApp] Jupyter Server 2.7.1 is running at:
[I 2023-09-04 11:30:48.997 ServerApp] http://jupyter-admin:8888/jupyterhub/user/admin/lab?token=...
[I 2023-09-04 11:30:48.997 ServerApp]     http://127.0.0.1:8888/jupyterhub/user/admin/lab?token=...
[I 2023-09-04 11:30:48.997 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2023-09-04 11:30:49.036 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[I 2023-09-04 11:30:49.940 ServerApp] 302 GET /jupyterhub/user/admin/ -> /jupyterhub/user/admin/lab? (@10.42.4.229) 0.75ms


# Clicked on the link with the token inside


[I 2023-09-04 11:31:09.091 ServerApp] 302 GET /jupyterhub/user/admin/?token=[secret] -> /jupyterhub/user/admin/lab?token=[secret] (@10.40.124.8) 1.00ms
[I 2023-09-04 11:31:09.142 ServerApp] 200 GET /jupyterhub/user/admin/lab?token=[secret] (admin@10.40.124.8) 33.42ms
[W 2023-09-04 11:31:09.189 ServerApp] No Hub user identified for request
[W 2023-09-04 11:31:09.190 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:09.349 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:09.355 ServerApp] Token stored in cookie may have expired
[I 2023-09-04 11:31:09.416 ServerApp] 200 GET /jupyterhub/user/admin/api/me?1693827069406 (admin@10.40.124.8) 1.29ms
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[I 2023-09-04 11:31:09.732 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827069406 (admin@10.40.124.8) 316.11ms
[I 2023-09-04 11:31:09.895 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings?1693827069408 (admin@10.40.124.8) 161.41ms
[I 2023-09-04 11:31:09.896 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels?1693827069411 (admin@10.40.124.8) 155.50ms
[I 2023-09-04 11:31:09.897 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827069412 (admin@10.40.124.8) 156.32ms
[I 2023-09-04 11:31:09.898 ServerApp] 200 GET /jupyterhub/user/admin/api/terminals?1693827069412 (admin@10.40.124.8) 156.73ms
[I 2023-09-04 11:31:09.899 ServerApp] 200 GET /jupyterhub/user/admin/api/me?1693827069424 (admin@10.40.124.8) 157.23ms
[I 2023-09-04 11:31:09.900 ServerApp] 101 GET /jupyterhub/user/admin/api/events/subscribe?token=[secret] (admin@10.40.124.8) 157.60ms
[I 2023-09-04 11:31:09.906 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827069745 (admin@10.40.124.8) 5.11ms
[I 2023-09-04 11:31:11.029 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/translations/default?1693827070995 (admin@10.40.124.8) 22.24ms
[I 2023-09-04 11:31:11.087 ServerApp] 200 GET /jupyterhub/user/admin/api/config/jupyterlabapputilsextensionannouncements?1693827071064 (admin@10.40.124.8) 14.21ms
[W 2023-09-04 11:31:11.089 LabApp] Could not determine jupyterlab build status without nodejs
[I 2023-09-04 11:31:11.090 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/build?1693827071078 (admin@10.40.124.8) 1.79ms
[I 2023-09-04 11:31:11.125 ServerApp] 200 GET /jupyterhub/user/admin/lsp/status?1693827071115 (admin@10.40.124.8) 3.06ms
[I 2023-09-04 11:31:11.195 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings/@jupyterlab/codemirror-extension:plugin?1693827071167 (admin@10.40.124.8) 20.74ms
[I 2023-09-04 11:31:11.196 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/translations?1693827071128 (admin@10.40.124.8) 60.26ms
[I 2023-09-04 11:31:11.215 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings/@jupyterlab/notebook-extension:panel?1693827071171 (admin@10.40.124.8) 18.01ms
[W 2023-09-04 11:31:11.215 ServerApp] Token stored in cookie may have expired
[I 2023-09-04 11:31:11.219 ServerApp] 200 GET /jupyterhub/user/admin/api/contents?content=1&1693827071133 (admin@10.40.124.8) 60.93ms
[I 2023-09-04 11:31:11.276 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/workspaces/default?1693827071260 (admin@10.40.124.8) 8.57ms
[I 2023-09-04 11:31:11.296 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb?content=0&1693827071285 (admin@10.40.124.8) 1.69ms
[I 2023-09-04 11:31:11.297 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb?content=0&1693827071286 (admin@10.40.124.8) 1.21ms
[I 2023-09-04 11:31:11.301 ServerApp] 200 GET /jupyterhub/user/admin/api/contents?content=1&1693827071280 (admin@10.40.124.8) 12.51ms
[I 2023-09-04 11:31:11.494 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings?ids_only=true&1693827071363 (admin@10.40.124.8) 22.59ms
[I 2023-09-04 11:31:12.067 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb?type=notebook&content=1&1693827071374 (admin@10.40.124.8) 585.54ms
[I 2023-09-04 11:31:12.536 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb?type=notebook&content=1&1693827071375 (admin@10.40.124.8) 1053.87ms
[I 2023-09-04 11:31:12.672 ServerApp] 204 PUT /jupyterhub/user/admin/lab/api/workspaces/default?1693827071913 (admin@10.40.124.8) 135.40ms
[I 2023-09-04 11:31:12.699 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb/checkpoints?1693827072101 (admin@10.40.124.8) 25.62ms
[I 2023-09-04 11:31:12.700 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb/checkpoints?1693827072101 (admin@10.40.124.8) 25.86ms
[I 2023-09-04 11:31:12.701 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb/checkpoints?1693827072104 (admin@10.40.124.8) 26.41ms
[I 2023-09-04 11:31:12.702 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb/checkpoints?1693827072565 (admin@10.40.124.8) 26.61ms
[I 2023-09-04 11:31:12.704 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb/checkpoints?1693827072566 (admin@10.40.124.8) 28.07ms
[I 2023-09-04 11:31:12.705 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb/checkpoints?1693827072571 (admin@10.40.124.8) 28.66ms
[I 2023-09-04 11:31:12.719 ServerApp] 200 GET /jupyterhub/user/admin/api/contents?content=1&1693827072699 (admin@10.40.124.8) 10.69ms
[I 2023-09-04 11:31:12.729 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827072710 (admin@10.40.124.8) 1.38ms
[I 2023-09-04 11:31:13.085 ServerApp] Kernel started: e0782be0-d968-4dee-9697-1489437ba2c4
[I 2023-09-04 11:31:13.088 ServerApp] 201 POST /jupyterhub/user/admin/api/sessions?1693827072740 (admin@10.40.124.8) 340.26ms
[I 2023-09-04 11:31:13.094 ServerApp] Kernel started: 6147e9e3-1a02-4ddd-a438-f0549582e779
[I 2023-09-04 11:31:13.096 ServerApp] 201 POST /jupyterhub/user/admin/api/sessions?1693827072741 (admin@10.40.124.8) 173.74ms
[I 2023-09-04 11:31:13.111 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827073099 (admin@10.40.124.8) 1.39ms
[I 2023-09-04 11:31:13.143 ServerApp] 200 PATCH /jupyterhub/user/admin/api/sessions/2978f17e-5e2c-4529-bc4a-c7d2c2fcbccc?1693827073132 (admin@10.40.124.8) 1.53ms
[I 2023-09-04 11:31:13.145 ServerApp] 200 PATCH /jupyterhub/user/admin/api/sessions/b015f5c5-8a45-4456-9de0-cdae0eb036c9?1693827073131 (admin@10.40.124.8) 1.15ms
[I 2023-09-04 11:31:13.146 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels?1693827073137 (admin@10.40.124.8) 1.02ms
[I 2023-09-04 11:31:13.164 ServerApp] 200 PATCH /jupyterhub/user/admin/api/sessions/2978f17e-5e2c-4529-bc4a-c7d2c2fcbccc?1693827073152 (admin@10.40.124.8) 1.96ms
[W 2023-09-04 11:31:13.168 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.168 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.183 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b5f952be-8d4e-4beb-868a-6e22e3bc43c5 (@10.40.124.8) 16.76ms
[I 2023-09-04 11:31:13.185 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827073154 (admin@10.40.124.8) 17.56ms
[I 2023-09-04 11:31:13.185 ServerApp] 200 PATCH /jupyterhub/user/admin/api/sessions/b015f5c5-8a45-4456-9de0-cdae0eb036c9?1693827073156 (admin@10.40.124.8) 18.01ms
[I 2023-09-04 11:31:13.192 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827073174 (admin@10.40.124.8) 5.70ms
[I 2023-09-04 11:31:13.210 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827073200 (admin@10.40.124.8) 0.93ms
[I 2023-09-04 11:31:13.217 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827073205 (admin@10.40.124.8) 4.25ms
[W 2023-09-04 11:31:13.229 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.229 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.230 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=714387b5-826a-4743-91f7-7f569a952a66 (@10.40.124.8) 1.54ms
[I 2023-09-04 11:31:13.247 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4?1693827073237 (admin@10.40.124.8) 2.02ms
[W 2023-09-04 11:31:13.278 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.278 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.280 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b6c1904f-9e16-48f3-a53e-ae2ff11907d8 (@10.40.124.8) 2.86ms
[I 2023-09-04 11:31:13.297 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827073287 (admin@10.40.124.8) 1.13ms
[W 2023-09-04 11:31:13.334 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.335 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.339 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=e89e3854-cfe7-4298-8905-f36a90cdacae (@10.40.124.8) 4.75ms
[I 2023-09-04 11:31:13.357 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827073346 (admin@10.40.124.8) 1.09ms
[W 2023-09-04 11:31:13.391 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.391 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.392 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b5f952be-8d4e-4beb-868a-6e22e3bc43c5 (@10.40.124.8) 1.60ms
[I 2023-09-04 11:31:13.410 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827073399 (admin@10.40.124.8) 1.10ms
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[W 2023-09-04 11:31:13.447 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.448 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.449 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=714387b5-826a-4743-91f7-7f569a952a66 (@10.40.124.8) 1.79ms
[I 2023-09-04 11:31:13.467 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4?1693827073456 (admin@10.40.124.8) 1.40ms
[W 2023-09-04 11:31:13.575 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.576 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.577 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b6c1904f-9e16-48f3-a53e-ae2ff11907d8 (@10.40.124.8) 1.72ms
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[I 2023-09-04 11:31:13.595 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827073584 (admin@10.40.124.8) 1.13ms
[I 2023-09-04 11:31:13.609 ServerApp] 200 GET /jupyterhub/user/admin/api/nbconvert?1693827071322 (admin@10.40.124.8) 2278.22ms
[W 2023-09-04 11:31:13.661 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.661 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.662 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=e89e3854-cfe7-4298-8905-f36a90cdacae (@10.40.124.8) 1.70ms
[I 2023-09-04 11:31:13.678 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827073669 (admin@10.40.124.8) 1.53ms
[I 2023-09-04 11:31:13.880 ServerApp] 204 PUT /jupyterhub/user/admin/lab/api/workspaces/default?1693827073712 (admin@10.40.124.8) 158.09ms
[W 2023-09-04 11:31:13.881 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:13.882 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:13.882 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=714387b5-826a-4743-91f7-7f569a952a66 (@10.40.124.8) 1.59ms
[I 2023-09-04 11:31:13.902 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4?1693827073891 (admin@10.40.124.8) 1.35ms
[W 2023-09-04 11:31:14.057 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:14.057 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:14.058 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b5f952be-8d4e-4beb-868a-6e22e3bc43c5 (@10.40.124.8) 1.88ms
[I 2023-09-04 11:31:14.076 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827074065 (admin@10.40.124.8) 1.30ms
[W 2023-09-04 11:31:14.264 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:14.264 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:14.265 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=e89e3854-cfe7-4298-8905-f36a90cdacae (@10.40.124.8) 1.93ms
[I 2023-09-04 11:31:14.283 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827074272 (admin@10.40.124.8) 1.15ms
[W 2023-09-04 11:31:14.441 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:14.441 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:14.443 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b6c1904f-9e16-48f3-a53e-ae2ff11907d8 (@10.40.124.8) 2.11ms
[I 2023-09-04 11:31:14.460 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827074450 (admin@10.40.124.8) 1.13ms
[W 2023-09-04 11:31:14.534 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:14.534 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:14.535 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=714387b5-826a-4743-91f7-7f569a952a66 (@10.40.124.8) 1.74ms
[I 2023-09-04 11:31:14.555 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4?1693827074542 (admin@10.40.124.8) 1.11ms
[W 2023-09-04 11:31:14.785 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:14.785 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:14.786 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b5f952be-8d4e-4beb-868a-6e22e3bc43c5 (@10.40.124.8) 1.72ms
[I 2023-09-04 11:31:14.808 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827074798 (admin@10.40.124.8) 1.17ms
[W 2023-09-04 11:31:15.759 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:15.759 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:15.760 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=e89e3854-cfe7-4298-8905-f36a90cdacae (@10.40.124.8) 1.68ms
[I 2023-09-04 11:31:15.777 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827075767 (admin@10.40.124.8) 1.07ms
[W 2023-09-04 11:31:16.105 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:16.105 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:16.106 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b6c1904f-9e16-48f3-a53e-ae2ff11907d8 (@10.40.124.8) 1.73ms
[I 2023-09-04 11:31:16.126 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827076114 (admin@10.40.124.8) 1.07ms
[W 2023-09-04 11:31:19.103 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:19.103 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:19.104 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b6c1904f-9e16-48f3-a53e-ae2ff11907d8 (@10.40.124.8) 2.12ms
[I 2023-09-04 11:31:19.122 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827079111 (admin@10.40.124.8) 1.12ms
[I 2023-09-04 11:31:19.956 ServerApp] 200 GET /jupyterhub/user/admin/api/terminals?1693827079944 (admin@10.40.124.8) 1.11ms
[W 2023-09-04 11:31:20.418 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:20.419 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:20.420 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=e89e3854-cfe7-4298-8905-f36a90cdacae (@10.40.124.8) 1.83ms
[I 2023-09-04 11:31:20.438 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827080427 (admin@10.40.124.8) 1.11ms
[W 2023-09-04 11:31:21.807 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:21.807 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:21.808 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=714387b5-826a-4743-91f7-7f569a952a66 (@10.40.124.8) 1.74ms
[I 2023-09-04 11:31:21.822 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4?1693827081814 (admin@10.40.124.8) 1.11ms
[W 2023-09-04 11:31:22.474 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:22.474 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:22.476 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b5f952be-8d4e-4beb-868a-6e22e3bc43c5 (@10.40.124.8) 3.99ms
[I 2023-09-04 11:31:22.497 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779?1693827082484 (admin@10.40.124.8) 2.60ms
[I 2023-09-04 11:31:22.764 ServerApp] 200 GET /jupyterhub/user/admin/api/contents?content=1&1693827082736 (admin@10.40.124.8) 17.28ms
[I 2023-09-04 11:31:23.174 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels?1693827083160 (admin@10.40.124.8) 2.60ms
[I 2023-09-04 11:31:23.221 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827083209 (admin@10.40.124.8) 2.69ms
[W 2023-09-04 11:31:23.411 ServerApp] Token stored in cookie may have expired
[W 2023-09-04 11:31:23.411 ServerApp] Couldn't authenticate WebSocket connection
[W 2023-09-04 11:31:23.413 ServerApp] 403 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=714387b5-826a-4743-91f7-7f569a952a66 (@10.40.124.8) 3.54ms



# Refresh page to access jupyterhub login screen and login with the same user




[I 2023-09-04 11:31:31.048 ServerApp] Logged-in user {'admin': False, 'name': 'admin', 'kind': 'user', 'groups': [], 'session_id': 'a3dc7f8d7f834cf7bfffc490cd4e6c60', 'scopes': ['access:servers!server=admin/', 'read:users:groups!user=admin', 'read:users:name!user=admin']}
[W 2023-09-04 11:31:31.053 ServerApp] No Hub user identified for request
[W 2023-09-04 11:31:31.053 ServerApp] Token stored in cookie may have expired
[I 2023-09-04 11:31:31.053 ServerApp] 302 GET /jupyterhub/user/admin/oauth_callback?code=[secret]&state=[secret] -> /jupyterhub/user/admin/lab (@10.40.124.8) 17.81ms
[I 2023-09-04 11:31:31.083 ServerApp] 200 GET /jupyterhub/user/admin/lab (admin@10.40.124.8) 19.96ms
[I 2023-09-04 11:31:31.227 ServerApp] 200 GET /jupyterhub/user/admin/api/me?1693827091215 (admin@10.40.124.8) 1.51ms
[I 2023-09-04 11:31:31.244 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827091214 (admin@10.40.124.8) 16.27ms
[I 2023-09-04 11:31:31.427 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings?1693827091217 (admin@10.40.124.8) 197.94ms
[I 2023-09-04 11:31:31.428 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels?1693827091218 (admin@10.40.124.8) 190.96ms
[I 2023-09-04 11:31:31.431 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827091219 (admin@10.40.124.8) 2.54ms
[I 2023-09-04 11:31:31.433 ServerApp] 200 GET /jupyterhub/user/admin/api/terminals?1693827091219 (admin@10.40.124.8) 3.33ms
[I 2023-09-04 11:31:31.433 ServerApp] 200 GET /jupyterhub/user/admin/api/me?1693827091237 (admin@10.40.124.8) 3.73ms
[I 2023-09-04 11:31:31.437 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827091254 (admin@10.40.124.8) 7.60ms
[I 2023-09-04 11:31:32.578 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/translations/default?1693827092529 (admin@10.40.124.8) 38.16ms
[I 2023-09-04 11:31:32.592 ServerApp] 101 GET /jupyterhub/user/admin/api/events/subscribe?token=[secret] (admin@10.40.124.8) 1.08ms
[I 2023-09-04 11:31:32.624 ServerApp] 200 GET /jupyterhub/user/admin/api/config/jupyterlabapputilsextensionannouncements?1693827092603 (admin@10.40.124.8) 13.51ms
[W 2023-09-04 11:31:32.626 LabApp] Could not determine jupyterlab build status without nodejs
[I 2023-09-04 11:31:32.626 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/build?1693827092616 (admin@10.40.124.8) 1.54ms
[I 2023-09-04 11:31:32.653 ServerApp] 200 GET /jupyterhub/user/admin/lsp/status?1693827092643 (admin@10.40.124.8) 2.72ms
[I 2023-09-04 11:31:32.690 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/translations?1693827092656 (admin@10.40.124.8) 28.08ms
[I 2023-09-04 11:31:32.706 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings/@jupyterlab/codemirror-extension:plugin?1693827092685 (admin@10.40.124.8) 14.86ms
[I 2023-09-04 11:31:32.722 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings/@jupyterlab/notebook-extension:panel?1693827092687 (admin@10.40.124.8) 15.46ms
[I 2023-09-04 11:31:32.731 ServerApp] 200 GET /jupyterhub/user/admin/api/contents?content=1&1693827092660 (admin@10.40.124.8) 47.30ms
[I 2023-09-04 11:31:32.768 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/workspaces/default?1693827092751 (admin@10.40.124.8) 8.87ms
[I 2023-09-04 11:31:32.786 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb?content=0&1693827092777 (admin@10.40.124.8) 1.68ms
[I 2023-09-04 11:31:32.793 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb?content=0&1693827092777 (admin@10.40.124.8) 1.37ms
[I 2023-09-04 11:31:32.797 ServerApp] 200 GET /jupyterhub/user/admin/api/contents?content=1&1693827092772 (admin@10.40.124.8) 17.08ms
[I 2023-09-04 11:31:32.809 ServerApp] 101 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=0ebc4cef-56c5-468d-9298-d295ec27e6d3 (admin@10.40.124.8) 7.56ms
[I 2023-09-04 11:31:32.810 ServerApp] Connecting to kernel e0782be0-d968-4dee-9697-1489437ba2c4.
[I 2023-09-04 11:31:32.884 ServerApp] 200 GET /jupyterhub/user/admin/lab/api/settings?ids_only=true&1693827092856 (admin@10.40.124.8) 21.50ms
[I 2023-09-04 11:31:33.297 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb?type=notebook&content=1&1693827092867 (admin@10.40.124.8) 412.58ms
[I 2023-09-04 11:31:33.738 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb?type=notebook&content=1&1693827092867 (admin@10.40.124.8) 853.07ms
[I 2023-09-04 11:31:33.740 ServerApp] Starting buffering for e0782be0-d968-4dee-9697-1489437ba2c4:0ebc4cef-56c5-468d-9298-d295ec27e6d3
[I 2023-09-04 11:31:33.744 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels?1693827092913 (admin@10.40.124.8) 3.57ms
[I 2023-09-04 11:31:33.745 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827092912 (admin@10.40.124.8) 3.87ms
[I 2023-09-04 11:31:33.899 ServerApp] 204 PUT /jupyterhub/user/admin/lab/api/workspaces/default?1693827093408 (admin@10.40.124.8) 156.71ms
[I 2023-09-04 11:31:33.917 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb/checkpoints?1693827093331 (admin@10.40.124.8) 174.59ms
[I 2023-09-04 11:31:33.917 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb/checkpoints?1693827093336 (admin@10.40.124.8) 174.80ms
[I 2023-09-04 11:31:33.918 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled1.ipynb/checkpoints?1693827093330 (admin@10.40.124.8) 176.05ms
[I 2023-09-04 11:31:33.918 ServerApp] 101 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=b97cb099-707f-45a9-a43f-de2c3e88c344 (admin@10.40.124.8) 177.19ms
[I 2023-09-04 11:31:33.919 ServerApp] Connecting to kernel 6147e9e3-1a02-4ddd-a438-f0549582e779.
[I 2023-09-04 11:31:33.922 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb/checkpoints?1693827093770 (admin@10.40.124.8) 17.02ms
[I 2023-09-04 11:31:33.923 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb/checkpoints?1693827093764 (admin@10.40.124.8) 17.90ms
[I 2023-09-04 11:31:33.923 ServerApp] 200 GET /jupyterhub/user/admin/api/contents/Untitled.ipynb/checkpoints?1693827093763 (admin@10.40.124.8) 18.90ms
[I 2023-09-04 11:31:33.955 ServerApp] 200 GET /jupyterhub/user/admin/api/sessions?1693827093945 (admin@10.40.124.8) 1.49ms
[I 2023-09-04 11:31:33.962 ServerApp] Starting buffering for 6147e9e3-1a02-4ddd-a438-f0549582e779:b97cb099-707f-45a9-a43f-de2c3e88c344
[I 2023-09-04 11:31:33.964 ServerApp] 200 GET /jupyterhub/user/admin/api/contents?content=1&1693827093924 (admin@10.40.124.8) 12.36ms
[I 2023-09-04 11:31:33.966 ServerApp] 200 GET /jupyterhub/user/admin/api/kernels?1693827093957 (admin@10.40.124.8) 1.31ms
[I 2023-09-04 11:31:33.992 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827093965 (admin@10.40.124.8) 17.85ms
[I 2023-09-04 11:31:33.997 ServerApp] 200 GET /jupyterhub/user/admin/api/kernelspecs?1693827093967 (admin@10.40.124.8) 4.82ms
[I 2023-09-04 11:31:34.010 ServerApp] 101 GET /jupyterhub/user/admin/api/kernels/6147e9e3-1a02-4ddd-a438-f0549582e779/channels?session_id=1362ffa2-931d-4107-b23e-6dab106af7e1 (admin@10.40.124.8) 1.80ms
[I 2023-09-04 11:31:34.010 ServerApp] Connecting to kernel 6147e9e3-1a02-4ddd-a438-f0549582e779.
[I 2023-09-04 11:31:34.054 ServerApp] 101 GET /jupyterhub/user/admin/api/kernels/e0782be0-d968-4dee-9697-1489437ba2c4/channels?session_id=27b4c5c7-be26-4734-923c-3c75752c5681 (admin@10.40.124.8) 1.92ms

I’ve tracked it down to a bug in the new JupyterHub singleuser-server extension. If you set the JUPYTERHUB_SINGLEUSER_EXTENSION=0 environment variable in your single-user environments (e.g. c.Spawner.environment), then I think this will work.

update: patch fixing the bug

Hi,
Sorry for the delay I was in holyday.
I just tried to pass the environment variable but it did not fix the issue.

I tried to pass it using:

c.KubeSpawner.environment = {
    'JUPYTERHUB_SINGLEUSER_EXTENSION': '0'
}
c.Spawner.environment = {
    'JUPYTERHUB_SINGLEUSER_EXTENSION': '0'
}

(With quote and without)

And directly throught the charts with the extraEnv variable:

extraEnv: 
    JUPYTERHUB_SINGLEUSER_EXTENSION: '0'

I might have missed the right way to pass it to the application.

@ pgossafcbg Have you solved this issue? I met the same issue. Thanks~

Solved. The 2nd works for me. Thanks.

@minrk This workaround works on my side. However, are there any potential drawbacks using this workaround, comparing to the default singleuser-server extension? Thanks a lot.

However, are there any potential drawbacks using this workaround, comparing to the default singleuser-server extension?

Shouldn’t be. The two are meant to be the same. The extension implementation is ‘the future’, but both are present for now to give folks a fallback while we work out any kinks in the new implementation (like this one). This situation is exactly why both are still present. Only when we’re quite confident that everything is working and we don’t need to support notebook < 7 will we consider removing the pre-extension implementation. At which point, this env should be ignored with a warning and you shouldn’t see any change.