Hello,
Newly set up jupyterhub is running on AWS EC2 instance. However while trying log in through the jupyterhub portal receiving subjected error. Any leads would be helpful.
from yaml import dump as ydump
ModuleNotFoundError: No module named ‘yaml’
[I 2025-01-07 15:06:32.949 JupyterHub log:192] 302 GET /hub/spawn → /hub/spawn-pending/dgadm (dgadm@::ffff:X.X.X.X) 1009.60ms
[I 2025-01-07 15:06:33.074 JupyterHub pages:397] dgadm is pending spawn
[I 2025-01-07 15:06:33.074 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’82328f7c245040068124658025e66cca:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-07 15:06:33.080 JupyterHub log:192] 200 GET /hub/spawn-pending/dgadm (dgadm@::ffff:X.X.X.X) 13.30ms
Task exception was never retrieved
future: <Task finished name=‘Task-33’ coro=<BaseHandler.spawn_single_user() done, defined at /usr/local/lib/python3.11/site-packages/jupyterhub/handlers/base.py:1007> exception=HTTPError()>
Traceback (most recent call last):
File “/usr/local/lib/python3.11/site-packages/jupyterhub/handlers/base.py”, line 1215, in spawn_single_user
await gen.with_timeout(
TimeoutError: Timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.11/site-packages/jupyterhub/handlers/base.py”, line 1249, in spawn_single_user
raise web.HTTPError(
tornado.web.HTTPError: HTTP 500: Internal Server Error (Spawner failed to start [status=1]. The logs for dgadm may contain details.)
15:10:40.389 [ConfigProxy] info: 200 GET /api/routes
[W 2025-01-07 15:11:55.503 JupyterHub user:1055] dgadm’s server never showed up at http://127.0.0.1:49697/user/dgadm/ after 300 seconds. Giving up.
Please provide jupyterhub configuration (especially the chosen Spawner and Authenticator) and more information about how you have installed jupyterhub. It is always a good idea to include the startup logs of JupyterHub as well, which includes a lot of useful information, rather than just the final error.
This might be relevant:
but it’s only part of an error, so I can’t comment on where that’s happening. But it does indicate a missing dependency.
1 Like
Hi @minrk Here is the full log
[I 2025-01-08 18:55:03.770 JupyterHub app:3346] Running JupyterHub version 5.2.1
[I 2025-01-08 18:55:03.770 JupyterHub app:3376] Using Authenticator: jupyterhub.auth.PAMAuthenticator-5.2.1
[I 2025-01-08 18:55:03.770 JupyterHub app:3376] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-5.2.1
[I 2025-01-08 18:55:03.770 JupyterHub app:3376] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-5.2.1
/usr/local/lib/python3.11/site-packages/jupyter_events/schema.py:68: JupyterEventsVersionWarning:
[I 2025-01-08 18:55:03.778 JupyterHub app:1831] Loading cookie_secret from /opt/local/jupyter/jupyterhub_cookie_secret
[I 2025-01-08 18:55:03.898 JupyterHub proxy:556] Generating new CONFIGPROXY_AUTH_TOKEN
[I 2025-01-08 18:55:03.972 JupyterHub app:3416] Initialized 0 spawners in 0.011 seconds
[I 2025-01-08 18:55:03.979 JupyterHub metrics:373] Found 0 active users in the last ActiveUserPeriods.twenty_four_hours
[I 2025-01-08 18:55:03.980 JupyterHub metrics:373] Found 2 active users in the last ActiveUserPeriods.seven_days
[I 2025-01-08 18:55:03.981 JupyterHub metrics:373] Found 2 active users in the last ActiveUserPeriods.thirty_days
[W 2025-01-08 18:55:03.981 JupyterHub proxy:625] Found proxy pid file: /opt/local/jupyter/jupyterhub-proxy.pid
[W 2025-01-08 18:55:03.981 JupyterHub proxy:637] Proxy no longer running at pid=440998
[I 2025-01-08 18:55:03.982 JupyterHub proxy:752] Starting proxy @ https://:8000
18:55:04.977 [ConfigProxy] info: Proxying https://*:8000 to (no default)
18:55:04.981 [ConfigProxy] info: Proxy API at
[I 2025-01-08 18:55:06.379 JupyterHub app:3739] Hub API listening on
18:55:06.380 [ConfigProxy] info: 200 GET /api/routes
18:55:06.385 [ConfigProxy] info: 200 GET /api/routes
[I 2025-01-08 18:55:06.385 JupyterHub proxy:477] Adding route for Hub: / =>
18:55:06.390 [ConfigProxy] info: Adding route / →
18:55:06.391 [ConfigProxy] info: Route added / →
18:55:06.392 [ConfigProxy] info: 201 POST /api/routes/
[I 2025-01-08 18:55:06.392 JupyterHub app:3770] JupyterHub is now running at https://:8000
[I 2025-01-08 18:56:36.568 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’XRYFuayka9xLnsu3kzUCsjv9PF_m3FE3zY31k9d6P4Y=:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-08 18:56:36.570 JupyterHub log:192] 302 GET /hub/login? → /hub/spawn (dgadm@::ffff:X.X.X.X) 22.57ms
[I 2025-01-08 18:56:36.834 JupyterHub provider:661] Creating oauth client jupyterhub-user-dgadm
[I 2025-01-08 18:56:36.864 JupyterHub spawner:1837] Spawning bash -l -c jupyter-labhub
[I 2025-01-08 18:56:37.782 JupyterHub log:192] 302 GET /hub/spawn → /hub/spawn-pending/dgadm (dgadm@::ffff:X.X.X.X) 1006.98ms
[I 2025-01-08 18:56:37.897 JupyterHub pages:397] dgadm is pending spawn
[I 2025-01-08 18:56:37.898 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’3a5b6e911a9a43ca9092ecb7d560657b:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-08 18:56:37.958 JupyterHub log:192] 200 GET /hub/spawn-pending/dgadm (dgadm@::ffff:X.X.X.X) 75.07ms
Traceback (most recent call last):
File “/usr/local/bin/jupyter-labhub”, line 5, in
from jupyterlab.labhubapp import main
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyterlab/init.py”, line 8, in
from .handlers.announcements import (
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyterlab/handlers/announcements.py”, line 15, in
from jupyterlab_server.translation_utils import translator
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyterlab_server/init.py”, line 6, in
from .app import LabServerApp
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyterlab_server/app.py”, line 10, in
from jupyter_server.extension.application import ExtensionApp, ExtensionAppJinjaMixin
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyter_server/extension/application.py”, line 17, in
from jupyter_server.serverapp import ServerApp
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyter_server/serverapp.py”, line 40, in
from jupyter_events.logger import EventLogger
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyter_events/init.py”, line 3, in
from .logger import EVENTS_METADATA_VERSION, EventLogger
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyter_events/logger.py”, line 20, in
from .schema import SchemaType
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyter_events/schema.py”, line 17, in
from . import yaml
File “/export/home/dgadm/.local/lib/python3.11/site-packages/jupyter_events/yaml.py”, line 7, in
from yaml import dump as ydump
ModuleNotFoundError: No module named ‘yaml’
[I 2025-01-08 18:56:42.546 JupyterHub login:45] User logged out: dgadm
[I 2025-01-08 18:56:42.553 JupyterHub log:192] 302 GET /hub/logout → /hub/login (@::ffff:X.X.X.X) 14.02ms
[I 2025-01-08 18:56:42.646 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’None:XRYFuayka9xLnsu3kzUCsjv9PF_m3FE3zY31k9d6P4Y=’ {‘path’: ‘/hub/’, ‘max_age’: 3600}
[I 2025-01-08 18:56:42.660 JupyterHub log:192] 200 GET /hub/login (@::ffff:X.X.X.X) 14.21ms
Task exception was never retrieved
future: <Task finished name=‘Task-38’ coro=<BaseHandler.spawn_single_user() done, defined at /usr/local/lib/python3.11/site-packages/jupyterhub/handlers/base.py:1007> exception=HTTPError()>
Traceback (most recent call last):
File “/usr/local/lib/python3.11/site-packages/jupyterhub/handlers/base.py”, line 1215, in spawn_single_user
await gen.with_timeout(
TimeoutError: Timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.11/site-packages/jupyterhub/handlers/base.py”, line 1249, in spawn_single_user
raise web.HTTPError(
tornado.web.HTTPError: HTTP 500: Internal Server Error (Spawner failed to start [status=1]. The logs for dgadm may contain details.)
[I 2025-01-08 18:59:42.660 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’XRYFuayka9xLnsu3kzUCsjv9PF_m3FE3zY31k9d6P4Y=:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-08 18:59:42.660 JupyterHub base:973] User logged in: dgadm
[I 2025-01-08 18:59:42.661 JupyterHub log:192] 302 POST /hub/login?next= → /user/dgadm/ (dgadm@::ffff:X.X.X.X) 364.77ms
[I 2025-01-08 18:59:42.758 JupyterHub log:192] 302 GET /user/dgadm/ → /hub/user/dgadm/ (@::ffff:X.X.X.X) 0.80ms
[I 2025-01-08 18:59:42.855 JupyterHub log:192] 303 GET /hub/user/dgadm/ (dgadm@::ffff:X.X.X.X) 8.52ms
[I 2025-01-08 18:59:42.966 JupyterHub pages:397] dgadm is pending spawn
[I 2025-01-08 18:59:42.966 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’98189e4601cf47b999e26859e41643eb:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-08 18:59:42.968 JupyterHub log:192] 200 GET /hub/spawn-pending/dgadm?next=%2Fhub%2Fuser%2Fdgadm%2F (dgadm@::ffff:X.X.X.X) 16.68ms
19:00:06.396 [ConfigProxy] info: 200 GET /api/routes
Here is jupyterhub_config file . Other lines are commented lines that I have not put here.
c = get_config() #noqa
c.Authenticator.allow_all = True
c.Spawner.start_timeout = 300 # Increase the timeout value as needed
c.Spawner.http_timeout = 300 # Increase the HTTP timeout value as needed
import os
def create_dir_hook(spawner):
username = spawner.user.name # get the username
volume_path = os.path.join(‘/export/home/username’)
if not os.path.exists(volume_path):
# create a directory with umask 0755
# hub and container user must have the same UID to be writeable
# still readable by other users on the system
os.mkdir(volume_path, 0o700)
# now do whatever you think your user needs
# …
pass
attach the hook function to the spawner
c.Spawner.pre_spawn_hook = create_dir_hook
c.Spawner.default_url = ‘/lab’
c.Spawner.cmd = [‘jupyter-labhub’]
c.JupyterHub.ssl_key = ‘/opt/local/python_upgrade/jupyter/certs/XXXXXXX.key’
c.JupyterHub.ssl_cert = ‘/opt/local/python_upgrade/jupyter/certs/XXXXXXX.XXXX.XXXcom.cer’
c.LocalProcessSpawner.shell_cmd = [‘bash’, ‘-l’, ‘-c’]
Also have pyyaml installed on the RHEL instance
pip show pyyaml
Name: PyYAML
Version: 6.0.2
Summary: YAML parser and emitter for Python
Home-page: https://pyyaml.org/
Author: Kirill Simonov
Author-email: xi@resolvent.net
License: MIT
Location: /usr/local/lib64/python3.11/site-packages
Requires:
Required-by: dask, jupyter-events, pandas-profiling, pyaml, snowflake.core
Seems like your Python environments are mixing up. From the logs, I see that you have installed JupyterHub and JupyterLab in /export/home/dgadm/.local. If so, try changing the c.Spawner.cmd to c.Spawner.cmd = ["/export/home/dgadm/.local/bin/jupyterhub-singleuser"] which should pick up modules from correct environment.
I would suggest you to create a virtual environment to install all JupyterHub related dependencies and use that virtual environment to run JupyterHub and JupyterLab. Also use absolute paths to jupyterhub and jupyterhub-singleuser apps so that they will always target the correct environment.
1 Like
Thank you. I’m using root . The packages are installed in /usr/local/lib/python3.11/site-packages
I checked changing path once with c.Spawner.cmd = [“/usr/local/bin/jupyterhub-singleuser”] and now I get " 404 : Not Found You are requesting a page that does not exist! " while accessing the portal. Would be great if you could suggest.
Here is the log:
[I 2025-01-09 23:15:29.901 JupyterHub app:3770] JupyterHub is now running at https://:8000
[D 2025-01-09 23:15:29.902 JupyterHub app:3339] It took 2.278 seconds for the Hub to start
[D 2025-01-09 23:16:02.104 JupyterHub base:411] Refreshing auth for dgadm
[I 2025-01-09 23:16:02.104 JupyterHub login:45] User logged out: dgadm
[I 2025-01-09 23:16:02.121 JupyterHub log:192] 302 GET /hub/logout → /hub/login (@::ffff:X.X.X.X) 55.01ms
[I 2025-01-09 23:16:02.292 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’None:YAP6mec4lWmQjUhVMxMXlm0N8w2IkDUkilBN10s7aT8=’ {‘path’: ‘/hub/’, ‘max_age’: 3600}
[I 2025-01-09 23:16:02.382 JupyterHub log:192] 200 GET /hub/login (@::ffff:X.X.X.X) 92.70ms
[D 2025-01-09 23:16:02.645 JupyterHub log:192] 304 GET /hub/static/components/@fortawesome/fontawesome-free/webfonts/fa-solid-900.woff2 (@::ffff:X.X.X.X) 5.15ms
[D 2025-01-09 23:16:08.468 JupyterHub roles:326] Assigning default role to User dgadm
[D 2025-01-09 23:16:08.470 JupyterHub base:681] Setting cookie jupyterhub-session-id: {‘httponly’: True, ‘secure’: True, ‘path’: ‘/’}
[D 2025-01-09 23:16:08.470 JupyterHub base:685] Setting cookie for dgadm: jupyterhub-hub-login
[D 2025-01-09 23:16:08.470 JupyterHub base:681] Setting cookie jupyterhub-hub-login: {‘httponly’: True, ‘secure’: True, ‘path’: ‘/hub/’}
[I 2025-01-09 23:16:08.471 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’YAP6mec4lWmQjUhVMxMXlm0N8w2IkDUkilBN10s7aT8=:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-09 23:16:08.471 JupyterHub base:973] User logged in: dgadm
[D 2025-01-09 23:16:08.471 JupyterHub user:496] Creating <class ‘jupyterhub.spawner.LocalProcessSpawner’> for dgadm:
[I 2025-01-09 23:16:08.472 JupyterHub log:192] 302 POST /hub/login?next= → /hub/spawn (dgadm@::ffff:X.X.X.X) 533.85ms
[D 2025-01-09 23:16:08.647 JupyterHub scopes:1010] Checking access to /hub/spawn via scope servers!server=dgadm/
[D 2025-01-09 23:16:08.647 JupyterHub pages:216] Triggering spawn with default options for dgadm
[D 2025-01-09 23:16:08.647 JupyterHub base:1095] Initiating spawn for dgadm
[D 2025-01-09 23:16:08.647 JupyterHub base:1099] 0/100 concurrent spawns
[D 2025-01-09 23:16:08.647 JupyterHub base:1104] 0 active servers
[I 2025-01-09 23:16:08.735 JupyterHub provider:661] Creating oauth client jupyterhub-user-dgadm
[D 2025-01-09 23:16:08.765 JupyterHub user:913] Calling Spawner.start for dgadm
[I 2025-01-09 23:16:08.767 JupyterHub spawner:1837] Spawning bash -l -c /usr/local/bin/jupyterhub-singleuser
[D 2025-01-09 23:16:08.831 JupyterHub spawner:1475] Polling subprocess every 30s
[D 2025-01-09 23:16:08.841 JupyterHub utils:292] Waiting 300s for server at http://127.0.0.1:36421/user/dgadm/api
[I 2025-01-09 23:16:09.649 JupyterHub log:192] 302 GET /hub/spawn → /hub/spawn-pending/dgadm (dgadm@::ffff:X.X.X.X) 1008.19ms
[D 2025-01-09 23:16:09.789 JupyterHub scopes:1010] Checking access to /hub/spawn-pending/dgadm via scope servers!server=dgadm/
[I 2025-01-09 23:16:09.789 JupyterHub pages:397] dgadm is pending spawn
[D 2025-01-09 23:16:09.789 JupyterHub _xsrf_utils:155] xsrf id mismatch b’YAP6mec4lWmQjUhVMxMXlm0N8w2IkDUkilBN10s7aT8=:59b01260f5f74ec9b5ed84434f870af4’ != b’87dac76df88f410b9a5d624ec33e3c8b:59b01260f5f74ec9b5ed84434f870af4’
[I 2025-01-09 23:16:09.790 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’87dac76df88f410b9a5d624ec33e3c8b:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-09 23:16:09.796 JupyterHub log:192] 200 GET /hub/spawn-pending/dgadm (dgadm@::ffff:X.X.X.X) 16.35ms
[D 2025-01-09 23:16:10.043 JupyterHub log:192] 304 GET /hub/static/components/@fortawesome/fontawesome-free/webfonts/fa-solid-900.woff2 (@::ffff:X.X.X.X) 1.01ms
[D 2025-01-09 23:16:10.054 JupyterHub scopes:1010] Checking access to /hub/api/users/dgadm/server/progress via scope read:servers!server=dgadm/
[I 2025-01-09 23:16:11.259 JupyterHubSingleUser] Starting jupyterhub single-user server extension version 5.2.1
[I 2025-01-09 23:16:11.260 JupyterHubSingleUser] Using default url from environment $JUPYTERHUB_DEFAULT_URL: /lab
[I 2025-01-09 23:16:11.263 ServerApp] jupyterhub | extension was successfully linked.
[I 2025-01-09 23:16:11.275 ServerApp] Writing Jupyter server cookie secret to /export/home/dgadm/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2025-01-09 23:16:11.306 JupyterHub log:192] 200 GET /hub/api (@127.0.0.1) 0.69ms
[I 2025-01-09 23:16:11.307 JupyterHubSingleUser] Updating Hub with activity every 300 seconds
[I 2025-01-09 23:16:11.307 ServerApp] jupyterhub | extension was successfully loaded.
[I 2025-01-09 23:16:11.311 ServerApp] Serving notebooks from local directory: /export/home/dgadm
[I 2025-01-09 23:16:11.311 ServerApp] Jupyter Server 2.15.0 is running at:
[I 2025-01-09 23:16:11.311 ServerApp] http://127.0.0.1:36421/user/dgadm/lab?token=…
[I 2025-01-09 23:16:11.311 ServerApp] http://127.0.0.1:36421/user/dgadm/lab?token=…
[I 2025-01-09 23:16:11.311 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[D 2025-01-09 23:16:11.321 JupyterHub base:366] Recording first activity for <APIToken(‘5a22…’, user=‘dgadm’, client_id=‘jupyterhub’)>
[D 2025-01-09 23:16:11.332 JupyterHub scopes:1010] Checking access to /hub/api/users/dgadm/activity via scope users:activity!user=dgadm
[D 2025-01-09 23:16:11.337 JupyterHub users:1006] Activity for user dgadm: 2025-01-10T07:16:11.275736Z
[D 2025-01-09 23:16:11.337 JupyterHub users:1024] Activity on server dgadm/: 2025-01-10T07:16:11.275736Z
[I 2025-01-09 23:16:11.344 JupyterHub log:192] 200 POST /hub/api/users/dgadm/activity (dgadm@127.0.0.1) 26.35ms
[I 2025-01-09 23:16:11.548 ServerApp] 200 GET /user/dgadm/api (@127.0.0.1) 1.71ms
[D 2025-01-09 23:16:11.548 JupyterHub utils:328] Server at http://127.0.0.1:36421/user/dgadm/api responded in 2.71s
[D 2025-01-09 23:16:11.549 JupyterHub _version:73] jupyterhub and jupyterhub-singleuser both on version 5.2.1
[I 2025-01-09 23:16:11.549 JupyterHub base:1124] User dgadm took 2.901 seconds to start
[I 2025-01-09 23:16:11.549 JupyterHub proxy:331] Adding user dgadm to proxy /user/dgadm/ => http://127.0.0.1:36421
[D 2025-01-09 23:16:11.549 JupyterHub proxy:925] Proxy: Fetching POST http://127.0.0.1:8001/api/routes/user/dgadm
23:16:11.554 [ConfigProxy] info: Adding route /user/dgadm → http://127.0.0.1:36421
23:16:11.555 [ConfigProxy] info: Route added /user/dgadm → http://127.0.0.1:36421
23:16:11.559 [ConfigProxy] info: 201 POST /api/routes/user/dgadm
[I 2025-01-09 23:16:11.561 JupyterHub users:899] Server dgadm is ready
[I 2025-01-09 23:16:11.565 JupyterHub log:192] 200 GET /hub/api/users/dgadm/server/progress?_xsrf=[secret] (dgadm@::ffff:X.X.X.X) 1519.29ms
[D 2025-01-09 23:16:11.697 JupyterHub scopes:1010] Checking access to /hub/spawn-pending/dgadm via scope servers!server=dgadm/
[I 2025-01-09 23:16:11.698 JupyterHub log:192] 302 GET /hub/spawn-pending/dgadm → /user/dgadm/ (dgadm@::ffff:X.X.X.X) 10.44ms
[I 2025-01-09 23:16:11.821 ServerApp] 302 GET /user/dgadm/ → /user/dgadm/lab? (@::ffff:X.X.X.X) 0.73ms
[I 2025-01-09 23:16:11.989 ServerApp] Setting new xsrf cookie for b’87dac76df88f410b9a5d624ec33e3c8b:YAP6mec4lWmQjUhVMxMXlm0N8w2IkDUkilBN10s7aT8=’ {‘path’: ‘/user/dgadm/’, ‘max_age’: 3600}
[W 2025-01-09 23:16:12.032 ServerApp] 404 GET /user/dgadm/lab? (@::ffff:X.X.X.X) 44.62ms
[W 2025-01-09 23:18:20.349 ServerApp] 404 GET /user/dgadm/lab? (@::ffff:X.X.X.X) 6.13ms
[D 2025-01-09 23:20:29.904 JupyterHub proxy:925] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
23:20:29.907 [ConfigProxy] info: 200 GET /api/routes
[D 2025-01-09 23:20:29.917 JupyterHub proxy:392] Checking routes
[D 2025-01-09 23:21:24.021 JupyterHub base:411] Refreshing auth for dgadm
[D 2025-01-09 23:21:24.022 JupyterHub scopes:1010] Checking access to /hub/api/users/dgadm/activity via scope users:activity!user=dgadm
[D 2025-01-09 23:21:24.029 JupyterHub users:1011] Not updating activity for <User(dgadm 1/1 running)>: 2025-01-10T07:16:11.275736Z < 2025-01-10T07:18:20.511000Z
[D 2025-01-09 23:21:24.029 JupyterHub users:1032] Not updating server activity on dgadm/: 2025-01-10T07:16:11.275736Z < 2025-01-10T07:18:20.511000Z
[I 2025-01-09 23:21:24.029 JupyterHub log:192] 200 POST /hub/api/users/dgadm/activity (dgadm@127.0.0.1) 17.96ms
[W 2025-01-09 23:24:04.140 ServerApp] 404 GET /user/dgadm/lab? (@::ffff:X.X.X.X) 1.90ms
[W 2025-01-09 23:24:09.251 ServerApp] 404 GET /user/dgadm/lab? (@::ffff:X.X.X.X) 1.64ms
[I 2025-01-09 23:24:27.731 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b’87dac76df88f410b9a5d624ec33e3c8b:59b01260f5f74ec9b5ed84434f870af4’ {‘path’: ‘/hub/’}
[I 2025-01-09 23:24:27.732 JupyterHub log:192] 302 GET /hub/login? → /user/dgadm/ (dgadm@::ffff:X.X.X.X) 22.22ms
[I 2025-01-09 23:24:27.840 ServerApp] 302 GET /user/dgadm/ → /user/dgadm/lab? (@::ffff:X.X.X.X) 0.58ms
[W 2025-01-09 23:24:27.953 ServerApp] 404 GET /user/dgadm/lab? (@::ffff:X.X.X.X) 1.87ms
There are no obvious errors in the logs you posted. It seems to me that you dont have JupyterLab installed in /usr/local/lib/python3.11/site-packages. Could you verify it and ensure you have JupyterLab package installed in that environment?
Jupyterlab is already installed.
pip show jupyterlab
Name: jupyterlab
Version: 4.3.4
Summary: JupyterLab computational environment
Home-page: https://jupyter.org
Author:
Author-email: Jupyter Development Team jupyter@googlegroups.com
License: Copyright (c) 2015-2024 Project Jupyter Contributors
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
-
Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer. -
Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution. -
Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS”
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Semver File License
The semver.py file is from GitHub - podhmo/python-node-semver: python version of node-semver
which is licensed under the “MIT” license. See the semver.py file for details.
Location: /usr/local/lib/python3.11/site-packages
Requires: async-lru, httpx, ipykernel, jinja2, jupyter-core, jupyter-lsp, jupyter-server, jupyterlab-server, notebook-shim, packaging, setuptools, tornado, traitlets
Required-by: jupyter, notebook
This is the message
Here is list of packages I have installed:
pip freeze
absl-py==2.1.0
aenum==3.1.15
aiobotocore==2.16.0
aiohappyeyeballs==2.4.4
aiohttp==3.11.11
aioitertools==0.12.0
aiosignal==1.3.2
alembic==1.14.0
annotated-types==0.7.0
anyio==4.8.0
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asn1crypto==1.5.1
asttokens==3.0.0
astunparse==1.6.3
async-generator==1.10
async-lru==2.0.4
async-timeout==4.0.3
asynctest==0.13.0
atpublic==5.0
attrs==24.3.0
Automat==24.8.1
awswrangler==3.10.1
azure-common==1.1.28
azure-core==1.32.0
azure-storage-blob==12.24.0
babel==2.16.0
backcall==0.2.0
backoff==2.2.1
bcrypt==4.2.1
beautifulsoup4==4.12.3
black==24.10.0
black-nb==0.7
bleach==6.2.0
blis==1.1.0
boto3==1.35.81
botocore==1.35.81
Bottleneck==1.4.2
cached-property==2.0.1
cachetools==5.5.0
catalogue==2.0.10
catboost==1.2.7
certifi==2024.12.14
certipy==0.2.1
cffi==1.17.1
chardet==5.2.0
charset-normalizer==3.4.1
click==8.1.7
cloudpathlib==0.20.0
cloudpickle==3.1.0
cmake==3.31.2
colorama==0.4.6
comm==0.2.2
confection==0.1.5
configparser==7.1.0
configurable-http-proxy==0.3.0
constantly==23.10.4
contourpy==1.3.1
coverage==7.6.9
coveralls==4.0.1
cryptography==44.0.0
cycler==0.12.1
cymem==2.0.10
dask==2024.12.1
debugpy==1.8.11
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.9
docopt==0.6.2
docutils==0.21.2
entrypoints==0.4
et_xmlfile==2.0.0
Events==0.5
exceptiongroup==1.2.2
executing==2.1.0
fastapi==0.115.6
fastjsonschema==2.21.1
filelock==3.16.1
flatbuffers==24.3.25
fonttools==4.55.3
fqdn==1.5.1
frozenlist==1.5.0
fsspec==2024.10.0
future==1.0.0
gast==0.6.0
gensim==4.3.3
genson==1.3.0
gitdb==4.0.11
GitPython==3.1.43
google-api-core==2.24.0
google-api-python-client==2.156.0
google-auth==2.37.0
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.1
google-pasta==0.2.0
googleapis-common-protos==1.66.0
graphviz==0.20.3
greenlet==3.1.1
gremlinpython==3.7.3
grpcio==1.68.1
h11==0.14.0
h2o==3.46.0.6
h5py==3.12.1
htmlmin==0.1.12
httpcore==1.0.7
httplib2==0.22.0
httpx==0.27.2
hyperlink==21.0.0
hypothesis==6.122.4
idna==3.4
ImageHash==4.3.1
imbalanced-learn==0.12.4
imblearn==0.0
importlib_metadata==8.5.0
importlib_resources==6.4.5
incremental==24.7.2
iniconfig==2.0.0
install==1.3.5
ipykernel==6.29.5
ipython==8.31.0
ipython-genutils==0.2.0
ipywidgets==8.1.5
isodate==0.7.2
isoduration==20.11.0
jedi==0.19.2
Jinja2==3.1.5
jmespath==1.0.1
joblib==1.1.1
json5==0.10.0
jsonpath-ng==1.7.0
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
jupyter==1.1.1
jupyter-console==6.6.3
jupyter-events==0.11.0
jupyter-lsp==2.2.5
jupyter-server-mathjax==0.2.6
jupyter-telemetry==0.1.0
jupyter_client==8.6.3
jupyter_core==5.7.2
jupyter_server==2.15.0
jupyter_server_terminals==0.5.3
jupyterhub==5.2.1
jupyterlab==4.3.4
jupyterlab_git==0.50.2
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.3
jupyterlab_widgets==3.0.13
keras==3.7.0
keras-nightly==3.7.0.dev2024121903
Keras-Preprocessing==1.1.2
kiwisolver==1.4.7
langcodes==3.5.0
language_data==1.3.0
libclang==18.1.1
lightgbm==4.5.0
llvmlite==0.43.0
locket==1.0.0
lockfile==0.12.2
lxml==4.9.2
Mako==1.3.8
marisa-trie==1.2.1
Markdown==3.7
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib==3.10.0
matplotlib-inline==0.1.7
mdurl==0.1.2
missingno==0.5.2
mistune==3.1.0
ml-dtypes==0.4.1
mock==5.1.0
mod-wsgi==4.9.4
more-itertools==10.5.0
mpmath==1.3.0
msrest==0.7.1
multidict==6.1.0
multimethod==1.12
murmurhash==1.0.11
mypy-extensions==1.0.0
namex==0.0.8
nbclassic==1.1.0
nbclient==0.10.2
nbconvert==7.16.5
nbdime==4.0.2
nbformat==5.10.4
nest-asyncio==1.6.0
networkx==3.4.2
nlp==0.4.0
nltk==3.9.1
nodejs==0.1.1
notebook==7.3.2
notebook_shim==0.2.4
numba==0.60.0
numpy==1.26.4
nvidia-cublas-cu12==12.4.5.8
nvidia-cuda-cupti-cu12==12.4.127
nvidia-cuda-nvrtc-cu12==12.4.127
nvidia-cuda-runtime-cu12==12.4.127
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.2.1.3
nvidia-curand-cu12==10.3.5.147
nvidia-cusolver-cu12==11.6.1.9
nvidia-cusparse-cu12==12.3.1.170
nvidia-nccl-cu12==2.21.5
nvidia-nvjitlink-cu12==12.4.127
nvidia-nvtx-cu12==12.4.127
oauth2client==4.1.3
oauthlib==3.2.2
openpyxl==3.1.5
opensearch-py==2.8.0
opt_einsum==3.4.0
optional-django==0.1.0
optree==0.13.1
oracledb==2.5.1
oscrypto==1.3.0
overrides==7.7.0
packaging==24.2
pamela==1.2.0
pandas==2.2.3
pandas-profiling==3.2.0
pandocfilters==1.5.1
paramiko==3.5.0
parso==0.8.4
partd==1.4.2
pathlib_abc==0.1.1
pathspec==0.12.1
pathy==0.11.0
patsy==1.0.1
pbr==6.1.0
pexpect==4.9.0
pg8000==1.31.2
phik==0.12.4
pickleshare==0.7.5
pillow==11.0.0
pipdeptree==2.24.0
pkgutil_resolve_name==1.3.10
platformdirs==4.3.6
plotly==5.24.1
pluggy==1.5.0
ply==3.11
preshed==3.0.9
prettier==0.0.7
progressbar2==4.5.0
prometheus_client==0.21.1
prompt_toolkit==3.0.48
propcache==0.2.1
proto-plus==1.25.0
protobuf==5.29.2
psutil==6.1.1
psycopg2==2.9.3
ptyprocess==0.7.0
pure_eval==0.2.3
py==1.11.0
pyaml==24.12.1
pyarrow==18.1.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pycryptodomex==3.21.0
pydantic==2.10.4
pydantic_core==2.27.2
Pygments==2.19.1
PyJWT==2.10.1
PyMySQL==1.0.2
PyNaCl==1.5.0
pyodbc==5.2.0
pyOpenSSL==24.3.0
pyparsing==3.2.0
PyPDF2==3.0.1
pyrsistent==0.20.0
PySocks==1.7.1
pystache==0.6.6
pytesseract==0.3.13
pytest==8.3.4
pytest-cov==6.0.0
python-daemon==3.1.2
python-dateutil==2.9.0.post0
python-editor==1.0.4
python-json-logger==3.2.1
python-utils==3.9.1
pytz==2024.2
PyWavelets==1.8.0
pyxdg==0.28
PyYAML==6.0.2
pyzmq==26.2.0
qtconsole==5.6.1
QtPy==2.4.2
redshift-connector==2.1.4
referencing==0.35.1
regex==2024.11.6
requests==2.32.3
requests-aws4auth==1.3.1
requests-oauthlib==2.0.0
requests-unixsocket==0.3.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.9.4
rpds-py==0.22.3
rsa==4.9
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.12
s3fs==2024.10.0
s3transfer==0.10.4
scikit-learn==1.3.2
scikit-optimize==0.10.2
SciPy==1.10.1
scramp==1.4.5
seaborn==0.13.2
semantic-version==2.10.0
Send2Trash==1.8.3
shap==0.46.0
shellingham==1.5.4
simplejson==3.19.3
six==1.17.0
slicer==0.0.8
smart-open==6.4.0
smmap==5.0.1
sniffio==1.3.1
snowflake==1.0.2
snowflake-connector-python==3.12.4
snowflake-sqlalchemy==1.7.2
snowflake._legacy==1.0.0
snowflake.core==1.0.2
sortedcontainers==2.4.0
sosreport==3.2
soupsieve==2.6
spacy==3.8.3
spacy-legacy==3.0.12
spacy-loggers==1.0.5
spyder-kernels==3.0.2
SQLAlchemy==2.0.36
srsly==2.5.0
stack-data==0.6.3
starlette==0.41.3
statsmodels==0.14.4
sudospawner==0.5.2
sympy==1.13.1
tabpy==2.13.0
tabulate==0.9.0
tangled-up-in-unicode==0.2.0
tenacity==9.0.0
tensorboard==2.18.0
tensorboard-data-server==0.7.2
tensorboard-plugin-wit==1.8.1
tensorflow==2.18.0
tensorflow-estimator==2.15.0
tensorflow-io-gcs-filesystem==0.37.1
termcolor==2.5.0
terminado==0.18.1
testpath==0.6.0
textblob==0.18.0.post0
thinc==8.3.3
threadpoolctl==3.5.0
tiktoken==0.8.0
tinycss2==1.4.0
tokenize_rt==6.1.0
toml==0.10.2
tomli==2.2.1
tomlkit==0.13.2
toolz==1.0.0
torch==2.5.1
tornado==6.4.2
tqdm==4.67.1
traitlets==5.14.3
triton==3.1.0
Twisted==24.11.0
typed-ast==1.5.5
typer==0.15.1
types-python-dateutil==2.9.0.20241206
typing_extensions==4.12.2
tzdata==2024.2
uri-template==1.3.0
uritemplate==4.1.1
urllib3==2.3.0
visions==0.7.4
wasabi==1.1.3
wcwidth==0.2.13
weasel==0.4.1
webcolors==24.11.1
webencodings==0.5.1
websocket-client==1.8.0
Werkzeug==3.1.3
widgetsnbextension==4.0.13
witwidget==1.8.1
wrapt==1.17.0
wurlitzer==3.1.1
xeus-python==0.15.12
xeus-python-shell==0.5.0
xgboost==2.1.3
xlrd==2.0.1
XlsxWriter==3.2.0
xverse==1.0.5
xxhash==3.5.0
yarl==1.18.3
yellowbrick==1.5
zipp==3.21.0
zmq==0.0.0
zope.interface==7.2
Strange. According to the logs you posted, the single server has started. You get 404 when you dont have JupyterLab installed in the environment. My guess is that it is still looking for JupyterLab in a wrong place? Can you try removing c.LocalProcessSpawner.shell_cmd = [‘bash’, ‘-l’, ‘-c’] from JupyterHub config?
Still getting same error after removing removing c.LocalProcessSpawner.shell_cmd = [‘bash’, ‘-l’, ‘-c’]. When I check the path using jupyter lab path command I see following. Is there any other way to troubleshoot ?
Application directory: /usr/local/share/jupyter/lab
User Settings directory: /root/.jupyter/lab/user-settings
Workspaces directory: /root/.jupyter/lab/workspaces
Sorry! Ran out of ideas. I noticed that you have JupyterHub and JupyterLab packages installed in multiple places on your server. What I would suggest is to clean up all the packages everywhere, create a virtual environment and use it.
1 Like