Not connecting to Kernels on Jupyterhub

I installed jupyterhub using the commands outlined in Install JupyterHub and JupyterLab from the ground up — JupyterHub 1.2.0 documentation, and I could use it yesterday. Now, when i open a notebook or start a console, it says “Connecting” for some time and then “Disconnected”.
I don’t find any errors, and I don’t know what to do.
Is there a way to see what really happens? Jupyerhub also does not find an active user in --show-config.

This sounds like websockets aren’t working, which is likely something in the nginx configuration.

Can you share your environment and some logs from both the browser and jupyterhub?

I’m sorry for my noobness, but I don’t really know how to share these info. Can you maybe tell me which commands would render the logs that might be helpful?
I have already reinstalled everything, and after each install it works until I stop using it, then on the next startup it does not.

No worries!

For JuypyterHub logs, it will depend on how JupyterHub is run. Since you followed that tutorial which puts jupyterhub in systemd, it’s likely sudo journalctl -u jupyterhub.

You can see your jupyterhub configuration with:

jupyterhub --show-config

and redact any sensitive information.

For the environment, you can use pip list or conda list (whichever package manager you are using).

For the browser console, you can go to the developer menu and select “Show JavaScript Console” (the exact names of the menu and console vary slightly across browsers).

Sharing your nginx config may help. You may also try connecting directly to the Hub, bypassing nginx. Doing that would isolate the nginx config as the culprit (most likely, I think).

Are you also using that proxy to connect to JupyterHub? It may be worth checking with your proxy administrators to see if they support websockets. Some old proxies don’t.

Thanks for the answer! I redacted the post to remove sensitive information

You can see your jupyterhub configuration with:

jupyterhub --show-config


[I 2023-08-10 15:23:15.988 JupyterHub app:2859] Running JupyterHub version 4.0.1
[I 2023-08-10 15:23:15.988 JupyterHub app:2889] Using Authenticator: jupyterhub.auth.PAMAuthenticator-4.0.1
[I 2023-08-10 15:23:15.988 JupyterHub app:2889] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-4.0.1
[I 2023-08-10 15:23:15.988 JupyterHub app:2889] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-4.0.1
[I 2023-08-10 15:23:15.995 JupyterHub app:1709] Writing cookie_secret to /home/t141g/work/25/fd1393196b51c109e244d91fd411b6/jupyterhub_cookie_secret
[I 2023-08-10 15:23:16.062 alembic.runtime.migration migration:213] Context impl SQLiteImpl.
[I 2023-08-10 15:23:16.062 alembic.runtime.migration migration:216] Will assume non-transactional DDL.
[I 2023-08-10 15:23:16.072 alembic.runtime.migration migration:621] Running stamp_revision  -> 0eee8c825d24
[I 2023-08-10 15:23:16.217 JupyterHub proxy:556] Generating new CONFIGPROXY_AUTH_TOKEN
[I 2023-08-10 15:23:16.247 JupyterHub app:1984] Not using allowed_users. Any authenticated user will be allowed.
[I 2023-08-10 15:23:16.262 JupyterHub app:2928] Initialized 0 spawners in 0.002 seconds
[I 2023-08-10 15:23:16.267 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.twenty_four_hours
[I 2023-08-10 15:23:16.268 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.seven_days
[I 2023-08-10 15:23:16.268 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.thirty_days

In the nginx config () I added the following three parts:
Before the server block:

map $http_upgrade $connection_upgrade {
          default upgrade;
          '' close;
     location /jupyter/ {
     # NOTE important to also set base url of jupyterhub to /jupyter in its config

     proxy_redirect   off;
     proxy_set_header X-Real-IP $remote_addr;
     proxy_set_header Host $host;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Forwarded-Proto $scheme;

     # websocket headers
     proxy_set_header Upgrade $http_upgrade;
     proxy_set_header Connection $connection_upgrade;

location /rstudio/ {
            rewrite ^/rstudio/(.*)$ /$1 break;
            proxy_pass http://localhost:port;
            proxy_redirect ~^http://localhost:port/(.*)$ $scheme://$host/rstudio/$1;

            # WebSocket support
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_read_timeout 86400;

The RStudio part I commented out when the issue arose, so it should be inactive now.

I need to use the system’s proxy to access the internet from Jupyter, so the first command I ran was:

import os

The environments accessible to Jupyter were set up via ~/environments/my_env/bin/python -m ipykernel install --user --name ‘python-my-env-new’ --display-name “Python New Env”. I don’t know how to show the installed packages because I only used them from within Jupyter.
I hope this helps :slight_smile: Thanks!

I don’t see anything obviously wrong, but if you can find a way to connect directly without using the proxy, that should isolate what’s wrong here. My guess is that your proxy doesn’t support http.

Logs from nginx and/or singeluser-servers might help. Making requests that isolate each component (e.g. direct to nginx, bypassing nginx) will help as well.