Cannot connect to the server, what am I missing?

JupyterHub
1

I’m trying to set up a JupyterHub server on my machine. The process is as follows:

  1. I build a docker image docker build my_image:comm . with the following content:
FROM ubuntu:22.04
RUN apt update
RUN apt install curl -y
RUN apt-get install nodejs -y
RUN apt-get install npm -y
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-host x86_64-unknown-linux-gnu --default-toolchain stable
RUN . /root/.cargo/env
RUN apt-get install python3 -y
RUN apt install python3-pip -y
RUN pip3 install -i https://pypi.mirrors.ustc.edu.cn/simple/ jupyterlab
RUN pip3 install -i https://pypi.mirrors.ustc.edu.cn/simple/ jupyterhub
RUN npm install -g configurable-http-proxy
RUN /root/.cargo/bin/cargo install evcxr_jupyter
RUN /root/.cargo/bin/evcxr_jupyter --install
CMD ["jupyter", "lab", "--allow-root"]
  1. I generate a config file jupyterhub_config.py:
c = get_config()
c.JupyterHub.spawner_class = "dockerspawner.DockerSpawner"
c.DockerSpawner.image = "my_image:com"
  1. I run the the command jupyterhub -f jupyterhub_config.py.

What I’m imagining is that, when I open the URL, the server will start a container of my_image:com, and this container server as a jupyterlab.
But when I open URL localhost:8000, I got this error:

W 2023-12-07 13:41:44.230 JupyterHub user:933] root's server never showed up at http://127.0.0.1:32774/user/root/ after 30 seconds. Giving up.
    
    Common causes of this timeout, and debugging tips:
    
    1. The server didn't finish starting,
       or it crashed due to a configuration issue.
       Check the single-user server's logs for hints at what needs fixing.
    2. The server started, but is not accessible at the specified URL.
       This may be a configuration issue specific to your chosen Spawner.
       Check the single-user server logs and resource to make sure the URL
       is correct and accessible from the Hub.
    3. (unlikely) Everything is working, but the server took too long to respond.
       To fix: increase `Spawner.http_timeout` configuration
       to a number of seconds that is enough for servers to become responsive.
    
[I 2023-12-07 13:41:44.233 JupyterHub dockerspawner:1424] Stopping container jupyter-root (id: c400779)
[E 2023-12-07 13:41:44.717 JupyterHub gen:630] Exception in Future <Task finished name='Task-22' coro=<BaseHandler.spawn_single_user.<locals>.finish_user_spawn() done, defined at /usr/local/lib/python3.10/dist-packages/jupyterhub/handlers/base.py:981> exception=TimeoutError("Server at http://127.0.0.1:32774/user/root/ didn't respond in 30 seconds")> after timeout
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/tornado/gen.py", line 625, in error_callback
        future.result()
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/handlers/base.py", line 988, in finish_user_spawn
        await spawn_future
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 914, in spawn
        await self._wait_up(spawner)
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 958, in _wait_up
        raise e
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 928, in _wait_up
        resp = await server.wait_up(
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/utils.py", line 289, in wait_for_http_server
        re = await exponential_backoff(
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/utils.py", line 237, in exponential_backoff
        raise asyncio.TimeoutError(fail_message)
    asyncio.exceptions.TimeoutError: Server at http://127.0.0.1:32774/user/root/ didn't respond in 30 seconds

I think it is due to the lack of connection between the jupyterlab from the container and jupyterhub running on my machine. What should I do?

2

There are generally two things you need to use jupyterhub in a container:

  1. the hub must listen on a connectable ip, e.g. c.JupyterHub.hub_ip = '0.0.0.0'
  2. the server command should be jupyterhub-singleuser, not jupyter lab .... This can be set in the image or via c.Spawner.cmd = "jupyterhub-singleuser"

And you almost certainly don’t want to allow the user containers to run as root, so creating a user in the image to run as is a good idea.

1 Like
4

Thanks for your reply. I’ve changed the files to as you said above, then I got this error:

[E 2023-12-07 19:06:07.938 JupyterHub gen:630] Exception in Future <Task finished name='Task-22' coro=<BaseHandler.spawn_single_user.<locals>.finish_user_spawn() done, defined at /usr/local/lib/python3.10/dist-packages/jupyterhub/handlers/base.py:981> exception=TimeoutError("Server at http://127.0.0.1:32779/user/root/ didn't respond in 300 seconds")> after timeout
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/tornado/gen.py", line 625, in error_callback
        future.result()
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/handlers/base.py", line 988, in finish_user_spawn
        await spawn_future
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 914, in spawn
        await self._wait_up(spawner)
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 958, in _wait_up
        raise e
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 928, in _wait_up
        resp = await server.wait_up(
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/utils.py", line 289, in wait_for_http_server
        re = await exponential_backoff(
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/utils.py", line 237, in exponential_backoff
        raise asyncio.TimeoutError(fail_message)
    asyncio.exceptions.TimeoutError: Server at http://127.0.0.1:32779/user/root/ didn't respond in 300 seconds

It stuck in connection, and I checked the version of jupyterhub both in my machine and the image, they are all the version 4.0.2. Do you have any idea about this error?

5

Try to share more logs, that could help. Start jupyterhub with jupyterhub --debug to get more logging info. Make sure to also include logs from the container itself.

Is the Hub also in a container, or on the host?

7

The hub on the host severed directly on my machine (not via docker image). I run the jupyterhub with command arg --debug, and got the terminal infomation as follows:

root@DESKTOP-L60P4Q8:~/my_docker/main# jupyterhub --debug -f ../jupyterhub_config.py
[D 2023-12-07 23:00:23.417 JupyterHub application:908] Looking for ../jupyterhub_config in /root/my_docker/main
[D 2023-12-07 23:00:23.417 JupyterHub application:929] Loaded config file: /root/my_docker/jupyterhub_config.py
[I 2023-12-07 23:00:23.419 JupyterHub app:2859] Running JupyterHub version 4.0.2
[I 2023-12-07 23:00:23.419 JupyterHub app:2889] Using Authenticator: jupyterhub.auth.PAMAuthenticator-4.0.2
[I 2023-12-07 23:00:23.419 JupyterHub app:2889] Using Spawner: dockerspawner.dockerspawner.DockerSpawner-13.0.0
[I 2023-12-07 23:00:23.419 JupyterHub app:2889] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-4.0.2
[D 2023-12-07 23:00:23.422 JupyterHub app:2816] Could not load pycurl: No module named 'pycurl'
    pycurl is recommended if you have a large number of users.
[I 2023-12-07 23:00:23.422 JupyterHub app:1664] Loading cookie_secret from /root/my_docker/main/jupyterhub_cookie_secret
[D 2023-12-07 23:00:23.422 JupyterHub app:1833] Connecting to db: sqlite:///jupyterhub.sqlite
[D 2023-12-07 23:00:23.430 JupyterHub orm:1016] database schema version found: 0eee8c825d24
[I 2023-12-07 23:00:23.450 JupyterHub proxy:556] Generating new CONFIGPROXY_AUTH_TOKEN
[D 2023-12-07 23:00:23.450 JupyterHub app:2104] Loading roles into database
[I 2023-12-07 23:00:23.454 JupyterHub app:1984] Not using allowed_users. Any authenticated user will be allowed.
[D 2023-12-07 23:00:23.456 JupyterHub app:2343] Purging expired APITokens
[D 2023-12-07 23:00:23.457 JupyterHub app:2343] Purging expired OAuthCodes
[D 2023-12-07 23:00:23.459 JupyterHub app:2179] Loading role assignments from config
[D 2023-12-07 23:00:23.464 JupyterHub app:2502] Initializing spawners
[D 2023-12-07 23:00:23.465 JupyterHub app:2633] Loaded users:
    
[I 2023-12-07 23:00:23.465 JupyterHub app:2928] Initialized 0 spawners in 0.002 seconds
[W 2023-12-07 23:00:23.466 JupyterHub app:2663] Deleting OAuth client jupyterhub-user-root
[I 2023-12-07 23:00:23.485 JupyterHub metrics:278] Found 1 active users in the last ActiveUserPeriods.twenty_four_hours
[I 2023-12-07 23:00:23.485 JupyterHub metrics:278] Found 1 active users in the last ActiveUserPeriods.seven_days
[I 2023-12-07 23:00:23.486 JupyterHub metrics:278] Found 1 active users in the last ActiveUserPeriods.thirty_days
[W 2023-12-07 23:00:23.486 JupyterHub proxy:746] Running JupyterHub without SSL.  I hope there is SSL termination happening somewhere else...
[I 2023-12-07 23:00:23.486 JupyterHub proxy:750] Starting proxy @ http://:8000
[D 2023-12-07 23:00:23.486 JupyterHub proxy:751] Proxy cmd: ['configurable-http-proxy', '--ip', '', '--port', '8000', '--api-ip', '127.0.0.1', '--api-port', '8001', '--error-target', 'http://DESKTOP-L60P4Q8:8081/hub/error', '--log-level', 'info']
[D 2023-12-07 23:00:23.487 JupyterHub proxy:670] Writing proxy pid file: jupyterhub-proxy.pid
23:00:23.597 [ConfigProxy] info: Proxying http://*:8000 to (no default)
23:00:23.598 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
[D 2023-12-07 23:00:23.633 JupyterHub proxy:787] Proxy started and appears to be up
[D 2023-12-07 23:00:23.634 JupyterHub proxy:880] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
23:00:23.638 [ConfigProxy] info: 200 GET /api/routes 
[I 2023-12-07 23:00:23.638 JupyterHub app:3178] Hub API listening on http://0.0.0.0:8081/hub/
[I 2023-12-07 23:00:23.638 JupyterHub app:3180] Private Hub API connect url http://DESKTOP-L60P4Q8:8081/hub/
[D 2023-12-07 23:00:23.638 JupyterHub proxy:389] Fetching routes to check
[D 2023-12-07 23:00:23.638 JupyterHub proxy:880] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
23:00:23.639 [ConfigProxy] info: 200 GET /api/routes 
[D 2023-12-07 23:00:23.639 JupyterHub proxy:392] Checking routes
[I 2023-12-07 23:00:23.639 JupyterHub proxy:477] Adding route for Hub: / => http://DESKTOP-L60P4Q8:8081
[D 2023-12-07 23:00:23.639 JupyterHub proxy:880] Proxy: Fetching POST http://127.0.0.1:8001/api/routes/
23:00:23.640 [ConfigProxy] info: Adding route / -> http://DESKTOP-L60P4Q8:8081
23:00:23.641 [ConfigProxy] info: Route added / -> http://DESKTOP-L60P4Q8:8081
23:00:23.641 [ConfigProxy] info: 201 POST /api/routes/ 
[I 2023-12-07 23:00:23.641 JupyterHub app:3245] JupyterHub is now running at http://:8000
[D 2023-12-07 23:00:23.641 JupyterHub app:2852] It took 0.226 seconds for the Hub to start
[D 2023-12-07 23:00:26.397 JupyterHub base:344] Refreshing auth for root
[D 2023-12-07 23:00:26.399 JupyterHub scopes:877] Checking access to /hub/api/users/root/server/progress via scope read:servers
[D 2023-12-07 23:00:26.399 JupyterHub scopes:690] Argument-based access to /hub/api/users/root/server/progress via read:servers
[W 2023-12-07 23:00:26.400 JupyterHub log:191] 404 GET /hub/api/users/root/server/progress?_xsrf=[secret] (root@::1) 6.25ms
[D 2023-12-07 23:00:39.255 JupyterHub scopes:877] Checking access to /hub/spawn-pending/root via scope servers
[D 2023-12-07 23:00:39.255 JupyterHub scopes:690] Argument-based access to /hub/spawn-pending/root via servers
[D 2023-12-07 23:00:39.255 JupyterHub user:431] Creating <class 'dockerspawner.dockerspawner.DockerSpawner'> for root:
[D 2023-12-07 23:00:39.255 JupyterHub dockerspawner:960] Loaded state for root: container name=jupyter-root, id=00795cf1a5f549fc242301dede05b3f02e09524d9d7467b489b0269ef2848101
[I 2023-12-07 23:00:39.269 JupyterHub log:191] 200 GET /hub/spawn-pending/root (root@::1) 15.60ms
23:00:39.287 [ConfigProxy] error: 503 GET /hub/static/favicon.ico socket hang up
[D 2023-12-07 23:00:39.289 JupyterHub log:191] 200 GET /hub/static/favicon.ico?v=fde5757cd3892b979919d3b1faa88a410f28829feb5ba22b6cf069f2c6c98675fceef90f932e49b510e74d65c681d5846b943e7f7cc1b41867422f0481085c1f (@::1) 2.13ms
[D 2023-12-07 23:00:39.290 JupyterHub pages:584] No template for 503
[I 2023-12-07 23:00:39.293 JupyterHub log:191] 200 GET /hub/error/503?url=%2Fhub%2Fstatic%2Ffavicon.ico%3Fv%3Dfde5757cd3892b979919d3b1faa88a410f28829feb5ba22b6cf069f2c6c98675fceef90f932e49b510e74d65c681d5846b943e7f7cc1b41867422f0481085c1f (@127.0.0.1) 3.42ms
[D 2023-12-07 23:00:39.301 JupyterHub log:191] 200 GET /hub/static/js/not_running.js?v=20231207230023 (@::1) 0.53ms
[D 2023-12-07 23:00:39.310 JupyterHub log:191] 200 GET /hub/static/js/utils.js?v=20231207230023 (@::1) 0.50ms
[D 2023-12-07 23:00:40.736 JupyterHub scopes:877] Checking access to /hub/spawn/root via scope servers
[D 2023-12-07 23:00:40.736 JupyterHub scopes:690] Argument-based access to /hub/spawn/root via servers
[D 2023-12-07 23:00:40.736 JupyterHub pages:217] Triggering spawn with default options for root
[D 2023-12-07 23:00:40.736 JupyterHub base:961] Initiating spawn for root
[D 2023-12-07 23:00:40.736 JupyterHub base:965] 0/100 concurrent spawns
[D 2023-12-07 23:00:40.736 JupyterHub base:970] 0 active servers
[I 2023-12-07 23:00:40.750 JupyterHub provider:659] Creating oauth client jupyterhub-user-root
[D 2023-12-07 23:00:40.764 JupyterHub user:794] Calling Spawner.start for root
[D 2023-12-07 23:00:40.772 JupyterHub dockerspawner:1027] Getting container 'jupyter-root'
[I 2023-12-07 23:00:40.773 JupyterHub dockerspawner:1320] Found existing container jupyter-root (id: 00795cf)
[I 2023-12-07 23:00:40.773 JupyterHub dockerspawner:1335] Starting container jupyter-root (id: 00795cf)
[I 2023-12-07 23:00:41.023 JupyterHub provider:661] Updating oauth client jupyterhub-user-root
[D 2023-12-07 23:00:41.028 JupyterHub spawner:1384] Polling subprocess every 30s
[D 2023-12-07 23:00:41.028 JupyterHub dockerspawner:972] Persisting state for root: container name=jupyter-root, id=00795cf1a5f549fc242301dede05b3f02e09524d9d7467b489b0269ef2848101
[I 2023-12-07 23:00:41.737 JupyterHub log:191] 302 GET /hub/spawn/root -> /hub/spawn-pending/root (root@::1) 1002.63ms
[D 2023-12-07 23:00:41.742 JupyterHub scopes:877] Checking access to /hub/spawn-pending/root via scope servers
[D 2023-12-07 23:00:41.742 JupyterHub scopes:690] Argument-based access to /hub/spawn-pending/root via servers
[I 2023-12-07 23:00:41.742 JupyterHub pages:398] root is pending spawn
[I 2023-12-07 23:00:41.744 JupyterHub log:191] 200 GET /hub/spawn-pending/root (root@::1) 3.56ms
23:00:41.769 [ConfigProxy] error: 503 GET /hub/static/favicon.ico socket hang up
[D 2023-12-07 23:00:41.770 JupyterHub log:191] 200 GET /hub/static/favicon.ico?v=fde5757cd3892b979919d3b1faa88a410f28829feb5ba22b6cf069f2c6c98675fceef90f932e49b510e74d65c681d5846b943e7f7cc1b41867422f0481085c1f (@::1) 0.58ms
[D 2023-12-07 23:00:41.771 JupyterHub pages:584] No template for 503
[I 2023-12-07 23:00:41.772 JupyterHub log:191] 200 GET /hub/error/503?url=%2Fhub%2Fstatic%2Ffavicon.ico%3Fv%3Dfde5757cd3892b979919d3b1faa88a410f28829feb5ba22b6cf069f2c6c98675fceef90f932e49b510e74d65c681d5846b943e7f7cc1b41867422f0481085c1f (@127.0.0.1) 0.92ms
[D 2023-12-07 23:00:41.778 JupyterHub scopes:877] Checking access to /hub/api/users/root/server/progress via scope read:servers
[D 2023-12-07 23:00:41.779 JupyterHub scopes:690] Argument-based access to /hub/api/users/root/server/progress via read:servers
[D 2023-12-07 23:00:50.738 JupyterHub dockerspawner:1027] Getting container 'jupyter-root'
[D 2023-12-07 23:00:50.741 JupyterHub dockerspawner:1012] Container 00795cf status: {'Dead': False,
     'Error': '',
     'ExitCode': 0,
     'FinishedAt': '2023-12-07T14:56:35.88423573Z',
     'OOMKilled': False,
     'Paused': False,
     'Pid': 1334570,
     'Restarting': False,
     'Running': True,
     'StartedAt': '2023-12-07T15:00:41.006255848Z',
     'Status': 'running'}
[W 2023-12-07 23:00:50.741 JupyterHub base:1122] User root is slow to become responsive (timeout=10)
[D 2023-12-07 23:00:50.741 JupyterHub base:1127] Expecting server for root at: http://127.0.0.1:32785/user/root/
[D 2023-12-07 23:01:11.031 JupyterHub dockerspawner:1027] Getting container 'jupyter-root'
[D 2023-12-07 23:01:11.037 JupyterHub dockerspawner:1012] Container 00795cf status: {'Dead': False,
     'Error': '',
     'ExitCode': 0,
     'FinishedAt': '2023-12-07T14:56:35.88423573Z',
     'OOMKilled': False,
     'Paused': False,
     'Pid': 1334570,
     'Restarting': False,
     'Running': True,
     'StartedAt': '2023-12-07T15:00:41.006255848Z',
     'Status': 'running'}
[D 2023-12-07 23:01:41.030 JupyterHub dockerspawner:1027] Getting container 'jupyter-root'
[D 2023-12-07 23:01:41.037 JupyterHub dockerspawner:1012] Container 00795cf status: {'Dead': False,
     'Error': '',
     'ExitCode': 1,
     'FinishedAt': '2023-12-07T15:01:28.120214187Z',
     'OOMKilled': False,
     'Paused': False,
     'Pid': 0,
     'Restarting': False,
     'Running': False,
     'StartedAt': '2023-12-07T15:00:41.006255848Z',
     'Status': 'exited'}

The last line of the terminal information shows that Status: exited, maybe this indicates that the sever did someting wrong while running?

Then I entered the image with command
docker run -dit --name my_container my_image:com
6279739b42d1989d7c666c23d1bbbf3594686689503ab6707bdd3d4e1180339a

docker exec -it my_container bash

And run jupyterhub-singleuser in the container:

root@6279739b42d1:/# jupyterhub-singleuser
[W 2023-12-07 15:11:57.035 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[W 2023-12-07 15:11:57.340 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-12-07 15:11:57.340 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2023-12-07 15:11:57.343 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-12-07 15:11:57.343 JupyterHubSingleUser] Starting jupyterhub single-user server extension version 4.0.2
[E 2023-12-07 15:11:57.343 JupyterHubSingleUser] Failed to load JupyterHubSingleUser server extension
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 274, in wrapped
        r = f(self, *args, **kwargs)
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 472, in load_config_file
        raise KeyError("Missing required environment $JUPYTERHUB_SERVICE_URL")
    KeyError: 'Missing required environment $JUPYTERHUB_SERVICE_URL'`

---------------update----------------
I run the jupyterhub, and tried to open the localhost:8000. So there start a container named jupyter-root, I print the log of this container with command docker logs jupyter-root, then I got this error:

[I 2023-12-07 11:31:10.032 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-12-07 11:31:10.146 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-12-07 11:31:10.188 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-12-07 11:31:10.190 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-12-07 11:31:10.190 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[E 2023-12-07 11:31:10.195 JupyterHubSingleUser] Failed to connect to my Hub at http://DESKTOP-L60P4Q8:8081/hub/api (attempt 1/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused
[I 2023-12-07 11:31:10.393 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[E 2023-12-07 11:31:12.211 JupyterHubSingleUser] Failed to connect to my Hub at http://DESKTOP-L60P4Q8:8081/hub/api (attempt 2/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2023-12-07 11:31:16.225 JupyterHubSingleUser] Failed to connect to my Hub at http://DESKTOP-L60P4Q8:8081/hub/api (attempt 3/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2023-12-07 11:31:24.243 JupyterHubSingleUser] Failed to connect to my Hub at http://DESKTOP-L60P4Q8:8081/hub/api (attempt 4/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2023-12-07 11:31:40.262 JupyterHubSingleUser] Failed to connect to my Hub at http://DESKTOP-L60P4Q8:8081/hub/api (attempt 5/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused

It seems the comes from connection problem.

Then I checked the JupyterHub host and found this log information on host terminal:

[D 2023-12-08 08:47:37.363 JupyterHub base:1127] Expecting server for root at: http://127.0.0.1:32787/user/root/

But there is an log information on the container’s terminal:

[E 2023-12-08 00:47:28.596 JupyterHubSingleUser] Failed to connect to my Hub at http://DESKTOP-L60P4Q8:8081/hub/api (attempt 1/5). Is it running?

Maybe it says something about the error?

8

I guess the docker single user container is not able to resolve http://DESKTOP-L60P4Q8:8081 to your host. Can you try with config c.JupyterHub.hub_bind_url = 'http://0.0.0.0:8081' in your JupyterHub config?

1 Like
10

I just tried this method out, and still got the same error.

11

Can you try with c.DockerSpawner.hub_connect_url = 'http://0.0.0.0:8081' ?

12

It still had error, but the error changed to this:

[I 2023-12-08 09:57:20.727 ServerApp] notebook | extension was successfully linked.
[I 2023-12-08 09:57:20.728 ServerApp] Writing Jupyter server cookie secret to /root/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2023-12-08 09:57:20.843 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-12-08 09:57:20.881 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-12-08 09:57:20.882 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-12-08 09:57:20.883 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[E 2023-12-08 09:57:20.886 JupyterHubSingleUser] Failed to connect to my Hub at http://0.0.0.0:8081/hub/api (attempt 1/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused
[I 2023-12-08 09:57:21.095 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[E 2023-12-08 09:57:22.892 JupyterHubSingleUser] Failed to connect to my Hub at http://0.0.0.0:8081/hub/api (attempt 2/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2023-12-08 09:57:26.897 JupyterHubSingleUser] Failed to connect to my Hub at http://0.0.0.0:8081/hub/api (attempt 3/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused
[E 2023-12-08 09:57:34.907 JupyterHubSingleUser] Failed to connect to my Hub at http://0.0.0.0:8081/hub/api (attempt 4/5). Is it running?
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/jupyterhub/singleuser/extension.py", line 336, in check_hub_version
        resp = await client.fetch(self.hub_auth.api_url)
    ConnectionRefusedError: [Errno 111] Connection refused

In the error information, the url had changed from http://DESKTOP-L60P4Q8:8081 to http://0.0.0.0:8081.

13

While JupyterHub is still running, can you start a single user container manually with a shell session and try to connect to hub API via curl -vvv -I http://0.0.0.0:8081/hub/api inside the container?

The error is saying that the single user container is not able to connect to Jupyterhub that is running on your host. You have to figure out what is blocking the communication.

15

I tried what you said as the following steps:

  1. I started the jupyterhub host on my machine with jupyterhub -f jupyterhub_config.py --debug
  2. I run the image that would be opened if I opened the url localhost:8000:
    docker run -dit --name my_container2 my_image2:com
  3. I entered the container my_container2:
    docker exec -it my_container2 bash.
  4. then inside the container, I run the command:
    curl -vvv -I http://0.0.0.0:8081/hub/api.
    This is the terminal information:
root@16f5251f29c4:/# curl -vvv -I http://0.0.0.0:8081
*   Trying 0.0.0.0:8081...
* connect to 0.0.0.0 port 8081 failed: Connection refused
* Failed to connect to 0.0.0.0 port 8081 after 0 ms: Connection refused
* Closing connection 0
curl: (7) Failed to connect to 0.0.0.0 port 8081 after 0 ms: Connection refused..
16

Seems like the services inside your container are not able to connect to services running on your host. You need to figure out why. Maybe this is a good starting point.

1 Like
17

Thanks very much for your detailed help, I’m going to make a hard try to it.