Hello manics,
thank you for your quick response and the tips on further debugging!
In the default setting everything works fine, I can even use a profile as long as I dont use the kubespawner_override.namespace
option.
Here are the logs from the profile+namespace setting with the debug logging:
Hub:
[I 2023-11-16 19:58:45.827 JupyterHub log:191] 200 GET /hub/login (@::ffff:10.244.120.64) 2.22ms
[D 2023-11-16 19:58:47.699 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 1.16ms
[D 2023-11-16 19:58:49.245 JupyterHub roles:281] Assigning default role to User b
[I 2023-11-16 19:58:49.248 JupyterHub roles:238] Adding role user for User: b
[D 2023-11-16 19:58:49.257 JupyterHub roles:281] Assigning default role to User b
[D 2023-11-16 19:58:49.260 JupyterHub base:587] Setting cookie jupyterhub-session-id: {'httponly': True, 'path': '/'}
[D 2023-11-16 19:58:49.260 JupyterHub base:591] Setting cookie for b: jupyterhub-hub-login
[D 2023-11-16 19:58:49.260 JupyterHub base:587] Setting cookie jupyterhub-hub-login: {'httponly': True, 'path': '/hub/'}
[I 2023-11-16 19:58:49.260 JupyterHub base:837] User logged in: b
[D 2023-11-16 19:58:49.260 JupyterHub user:431] Creating <class 'kubespawner.spawner.KubeSpawner'> for b:
[I 2023-11-16 19:58:49.263 JupyterHub log:191] 302 POST /hub/login?next= -> /hub/spawn (b@::ffff:10.244.120.64) 23.69ms
[D 2023-11-16 19:58:49.271 JupyterHub base:299] Recording first activity for <User(b 0/1 running)>
[D 2023-11-16 19:58:49.275 JupyterHub scopes:877] Checking access to /hub/spawn via scope servers
[D 2023-11-16 19:58:49.275 JupyterHub scopes:690] Argument-based access to /hub/spawn via servers
[D 2023-11-16 19:58:49.284 JupyterHub pages:209] Serving options form for b
[I 2023-11-16 19:58:49.285 JupyterHub log:191] 200 GET /hub/spawn (b@::ffff:10.244.120.64) 16.17ms
[D 2023-11-16 19:58:49.690 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.80ms
[D 2023-11-16 19:58:50.321 JupyterHub scopes:877] Checking access to /hub/spawn via scope servers
[D 2023-11-16 19:58:50.322 JupyterHub scopes:690] Argument-based access to /hub/spawn via servers
[D 2023-11-16 19:58:50.322 JupyterHub pages:257] Triggering spawn with supplied form options for b
[D 2023-11-16 19:58:50.322 JupyterHub base:961] Initiating spawn for b
[D 2023-11-16 19:58:50.322 JupyterHub base:965] 0/64 concurrent spawns
[D 2023-11-16 19:58:50.322 JupyterHub base:970] 1 active servers
[I 2023-11-16 19:58:50.335 JupyterHub provider:659] Creating oauth client jupyterhub-user-b
[D 2023-11-16 19:58:50.346 JupyterHub user:794] Calling Spawner.start for b
[D 2023-11-16 19:58:50.346 JupyterHub spawner:3199] Applying KubeSpawner override for profile '<big>Tensorflow 2.10 ohne GPU Support</big>'
[D 2023-11-16 19:58:50.346 JupyterHub spawner:3177] .. overriding KubeSpawner value extra_labels={'hub.jupyter.org/network-access-hub': 'true'}
[D 2023-11-16 19:58:50.346 JupyterHub spawner:3177] .. overriding KubeSpawner value namespace=other-namespace
[I 2023-11-16 19:58:50.347 JupyterHub log:191] 302 POST /hub/spawn?_xsrf=[secret] -> /hub/spawn-pending/b?_xsrf=[secret] (b@::ffff:10.244.120.64) 28.88ms
[I 2023-11-16 19:58:50.350 JupyterHub spawner:2497] Attempting to create pod jupyter-b, with timeout 3
[D 2023-11-16 19:58:50.354 JupyterHub scopes:877] Checking access to /hub/spawn-pending/b via scope servers
[D 2023-11-16 19:58:50.354 JupyterHub scopes:690] Argument-based access to /hub/spawn-pending/b via servers
[I 2023-11-16 19:58:50.354 JupyterHub pages:398] b is pending spawn
[I 2023-11-16 19:58:50.354 JupyterHub log:191] 200 GET /hub/spawn-pending/b?_xsrf=[secret] (b@::ffff:10.244.120.64) 2.02ms
[D 2023-11-16 19:58:50.387 JupyterHub scopes:877] Checking access to /hub/api/users/b/server/progress via scope read:servers
[D 2023-11-16 19:58:50.388 JupyterHub scopes:690] Argument-based access to /hub/api/users/b/server/progress via read:servers
[D 2023-11-16 19:58:50.388 JupyterHub spawner:2296] progress generator: jupyter-b
[D 2023-11-16 19:58:50.682 JupyterHub reflector:374] pods watcher timeout
[D 2023-11-16 19:58:50.682 JupyterHub reflector:289] Connecting pods watcher
[I 2023-11-16 19:58:51.538 JupyterHub log:191] 200 GET /hub/api (@10.244.120.78) 0.34ms
[D 2023-11-16 19:58:51.551 JupyterHub base:299] Recording first activity for <APIToken('6195...', user='b', client_id='jupyterhub')>
[D 2023-11-16 19:58:51.555 JupyterHub scopes:877] Checking access to /hub/api/users/b/activity via scope users:activity
[D 2023-11-16 19:58:51.555 JupyterHub scopes:690] Argument-based access to /hub/api/users/b/activity via users:activity
[D 2023-11-16 19:58:51.555 JupyterHub users:874] Activity for user b: 2023-11-16T19:58:51.528437Z
[D 2023-11-16 19:58:51.555 JupyterHub users:892] Activity on server b/: 2023-11-16T19:58:51.528437Z
[I 2023-11-16 19:58:51.557 JupyterHub log:191] 200 POST /hub/api/users/b/activity (b@10.244.120.78) 7.05ms
[D 2023-11-16 19:58:51.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.45ms
[D 2023-11-16 19:58:53.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.42ms
[D 2023-11-16 19:58:55.697 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 3.30ms
[D 2023-11-16 19:58:57.699 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 2.07ms
[D 2023-11-16 19:58:59.691 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.61ms
[D 2023-11-16 19:59:00.689 JupyterHub reflector:374] pods watcher timeout
[D 2023-11-16 19:59:00.690 JupyterHub reflector:289] Connecting pods watcher
[D 2023-11-16 19:59:01.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.41ms
[D 2023-11-16 19:59:03.688 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.36ms
[D 2023-11-16 19:59:05.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.76ms
[D 2023-11-16 19:59:07.695 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 2.20ms
[D 2023-11-16 19:59:09.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.72ms
[D 2023-11-16 19:59:10.711 JupyterHub reflector:374] pods watcher timeout
[D 2023-11-16 19:59:10.711 JupyterHub reflector:289] Connecting pods watcher
[D 2023-11-16 19:59:11.693 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 2.50ms
[D 2023-11-16 19:59:13.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.45ms
[D 2023-11-16 19:59:15.701 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 3.26ms
[D 2023-11-16 19:59:17.698 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 2.02ms
[D 2023-11-16 19:59:19.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.50ms
[D 2023-11-16 19:59:20.733 JupyterHub reflector:374] pods watcher timeout
[D 2023-11-16 19:59:20.734 JupyterHub reflector:289] Connecting pods watcher
[D 2023-11-16 19:59:21.696 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 2.65ms
[D 2023-11-16 19:59:23.690 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.40ms
[D 2023-11-16 19:59:25.699 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 1.86ms
[D 2023-11-16 19:59:27.701 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 6.81ms
[D 2023-11-16 19:59:29.690 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.50ms
[D 2023-11-16 19:59:30.668 JupyterHub proxy:880] Proxy: Fetching GET http://proxy-api:8001/api/routes
[D 2023-11-16 19:59:30.684 JupyterHub proxy:392] Checking routes
[D 2023-11-16 19:59:30.746 JupyterHub reflector:374] pods watcher timeout
[D 2023-11-16 19:59:30.746 JupyterHub reflector:289] Connecting pods watcher
[D 2023-11-16 19:59:31.698 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 2.50ms
[D 2023-11-16 19:59:33.689 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 0.39ms
[D 2023-11-16 19:59:35.698 JupyterHub log:191] 200 GET /hub/health (@10.244.120.64) 2.29ms
User-Pod:
Defaulted container "notebook" out of: notebook, block-cloud-metadata (init)
[I 2023-11-16 19:58:51.295 ServerApp] Package jupyterhub took 0.0000s to import
[I 2023-11-16 19:58:51.301 ServerApp] Package jupyter_lsp took 0.0056s to import
[W 2023-11-16 19:58:51.301 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-11-16 19:58:51.304 ServerApp] Package jupyter_server_terminals took 0.0032s to import
[I 2023-11-16 19:58:51.345 ServerApp] Package jupyterlab took 0.0407s to import
[I 2023-11-16 19:58:51.367 ServerApp] Package nbclassic took 0.0008s to import
[W 2023-11-16 19:58:51.368 ServerApp] A `_jupyter_server_extension_points` function was not found in nbclassic. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-11-16 19:58:51.368 ServerApp] Package nbgitpuller took 0.0005s to import
[I 2023-11-16 19:58:51.368 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-11-16 19:58:51.368 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-11-16 19:58:51.369 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2023-11-16 19:58:51.371 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-11-16 19:58:51.371 JupyterHubSingleUser] Starting jupyterhub single-user server extension version 4.0.2
[I 2023-11-16 19:58:51.371 JupyterHubSingleUser] Using default url from server extension lab: /lab
[I 2023-11-16 19:58:51.372 ServerApp] jupyterhub | extension was successfully linked.
[W 2023-11-16 19:58:51.373 LabApp] 'extra_template_paths' was found in both NotebookApp and ServerApp. This is likely a recent change. This config will only be set in NotebookApp. Please check if you should also config these traits in ServerApp for your purpose.
[I 2023-11-16 19:58:51.375 ServerApp] jupyterlab | extension was successfully linked.
[W 2023-11-16 19:58:51.375 NotebookApp] 'extra_template_paths' was found in both NotebookApp and ServerApp. This is likely a recent change. This config will only be set in NotebookApp. Please check if you should also config these traits in ServerApp for your purpose.
[I 2023-11-16 19:58:51.376 ServerApp] nbclassic | extension was successfully linked.
[I 2023-11-16 19:58:51.376 ServerApp] nbgitpuller | extension was successfully linked.
[I 2023-11-16 19:58:51.377 ServerApp] Writing Jupyter server cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2023-11-16 19:58:51.523 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-11-16 19:58:51.532 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-11-16 19:58:51.533 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-11-16 19:58:51.534 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-11-16 19:58:51.538 JupyterHubSingleUser] Updating Hub with activity every 300 seconds
[I 2023-11-16 19:58:51.539 ServerApp] jupyterhub | extension was successfully loaded.
[I 2023-11-16 19:58:51.540 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.11/site-packages/jupyterlab
[I 2023-11-16 19:58:51.540 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
[I 2023-11-16 19:58:51.540 LabApp] Extension Manager is 'pypi'.
[I 2023-11-16 19:58:51.541 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-11-16 19:58:51.545 ServerApp] nbclassic | extension was successfully loaded.
[I 2023-11-16 19:58:51.546 ServerApp] nbgitpuller | extension was successfully loaded.
[I 2023-11-16 19:58:51.546 ServerApp] Serving notebooks from local directory: /home/jovyan
[I 2023-11-16 19:58:51.546 ServerApp] Jupyter Server 2.7.3 is running at:
[I 2023-11-16 19:58:51.546 ServerApp] http://jupyter-b:8888/user/b/lab?token=...
[I 2023-11-16 19:58:51.546 ServerApp] http://127.0.0.1:8888/user/b/lab?token=...
[I 2023-11-16 19:58:51.546 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2023-11-16 19:58:51.549 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
Also if I disable all network policies by setting all networkPolicy.enabled
settings in values.yaml
to false, I get the same results (or at least I didn’t see a difference):
Hub:
[I 2023-11-16 20:07:40.998 JupyterHub log:191] 200 GET /hub/login (@::ffff:10.244.0.1) 0.96ms
[D 2023-11-16 20:07:42.036 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.36ms
[D 2023-11-16 20:07:43.312 JupyterHub roles:281] Assigning default role to User c
[I 2023-11-16 20:07:43.321 JupyterHub roles:238] Adding role user for User: c
[D 2023-11-16 20:07:43.329 JupyterHub roles:281] Assigning default role to User c
[D 2023-11-16 20:07:43.330 JupyterHub base:587] Setting cookie jupyterhub-session-id: {'httponly': True, 'path': '/'}
[D 2023-11-16 20:07:43.330 JupyterHub base:591] Setting cookie for c: jupyterhub-hub-login
[D 2023-11-16 20:07:43.330 JupyterHub base:587] Setting cookie jupyterhub-hub-login: {'httponly': True, 'path': '/hub/'}
[I 2023-11-16 20:07:43.330 JupyterHub base:837] User logged in: c
[D 2023-11-16 20:07:43.330 JupyterHub user:431] Creating <class 'kubespawner.spawner.KubeSpawner'> for c:
[I 2023-11-16 20:07:43.331 JupyterHub log:191] 302 POST /hub/login?next= -> /hub/spawn (c@::ffff:10.244.0.1) 21.55ms
[D 2023-11-16 20:07:43.338 JupyterHub base:299] Recording first activity for <User(c 0/1 running)>
[D 2023-11-16 20:07:43.341 JupyterHub scopes:877] Checking access to /hub/spawn via scope servers
[D 2023-11-16 20:07:43.341 JupyterHub scopes:690] Argument-based access to /hub/spawn via servers
[D 2023-11-16 20:07:43.346 JupyterHub pages:209] Serving options form for c
[I 2023-11-16 20:07:43.347 JupyterHub log:191] 200 GET /hub/spawn (c@::ffff:10.244.0.1) 9.53ms
[D 2023-11-16 20:07:44.037 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.76ms
[D 2023-11-16 20:07:44.242 JupyterHub scopes:877] Checking access to /hub/spawn via scope servers
[D 2023-11-16 20:07:44.242 JupyterHub scopes:690] Argument-based access to /hub/spawn via servers
[D 2023-11-16 20:07:44.242 JupyterHub pages:257] Triggering spawn with supplied form options for c
[D 2023-11-16 20:07:44.242 JupyterHub base:961] Initiating spawn for c
[D 2023-11-16 20:07:44.242 JupyterHub base:965] 1/64 concurrent spawns
[D 2023-11-16 20:07:44.242 JupyterHub base:970] 1 active servers
[I 2023-11-16 20:07:44.252 JupyterHub provider:659] Creating oauth client jupyterhub-user-c
[D 2023-11-16 20:07:44.259 JupyterHub user:794] Calling Spawner.start for c
[D 2023-11-16 20:07:44.259 JupyterHub spawner:3199] Applying KubeSpawner override for profile '<big>Tensorflow 2.10 ohne GPU Support</big>'
[D 2023-11-16 20:07:44.259 JupyterHub spawner:3177] .. overriding KubeSpawner value extra_labels={'hub.jupyter.org/network-access-hub': 'true'}
[D 2023-11-16 20:07:44.259 JupyterHub spawner:3177] .. overriding KubeSpawner value namespace=other-namespace
[I 2023-11-16 20:07:44.259 JupyterHub log:191] 302 POST /hub/spawn?_xsrf=[secret] -> /hub/spawn-pending/c?_xsrf=[secret] (c@::ffff:10.244.0.1) 18.95ms
[I 2023-11-16 20:07:44.261 JupyterHub spawner:2497] Attempting to create pod jupyter-c, with timeout 3
[D 2023-11-16 20:07:44.264 JupyterHub scopes:877] Checking access to /hub/spawn-pending/c via scope servers
[D 2023-11-16 20:07:44.264 JupyterHub scopes:690] Argument-based access to /hub/spawn-pending/c via servers
[I 2023-11-16 20:07:44.264 JupyterHub pages:398] c is pending spawn
[I 2023-11-16 20:07:44.265 JupyterHub log:191] 200 GET /hub/spawn-pending/c?_xsrf=[secret] (c@::ffff:10.244.0.1) 1.59ms
[D 2023-11-16 20:07:44.303 JupyterHub scopes:877] Checking access to /hub/api/users/c/server/progress via scope read:servers
[D 2023-11-16 20:07:44.304 JupyterHub scopes:690] Argument-based access to /hub/api/users/c/server/progress via read:servers
[D 2023-11-16 20:07:44.304 JupyterHub spawner:2296] progress generator: jupyter-c
[D 2023-11-16 20:07:46.036 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.34ms
[I 2023-11-16 20:07:46.428 JupyterHub log:191] 200 GET /hub/api (@10.244.0.13) 0.32ms
[D 2023-11-16 20:07:46.443 JupyterHub base:299] Recording first activity for <APIToken('ad22...', user='c', client_id='jupyterhub')>
[D 2023-11-16 20:07:46.447 JupyterHub scopes:877] Checking access to /hub/api/users/c/activity via scope users:activity
[D 2023-11-16 20:07:46.447 JupyterHub scopes:690] Argument-based access to /hub/api/users/c/activity via users:activity
[D 2023-11-16 20:07:46.448 JupyterHub users:874] Activity for user c: 2023-11-16T20:07:46.416655Z
[D 2023-11-16 20:07:46.448 JupyterHub users:892] Activity on server c/: 2023-11-16T20:07:46.416655Z
[I 2023-11-16 20:07:46.450 JupyterHub log:191] 200 POST /hub/api/users/c/activity (c@10.244.0.13) 7.33ms
[D 2023-11-16 20:07:48.037 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.91ms
[D 2023-11-16 20:07:50.039 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.87ms
[D 2023-11-16 20:07:52.036 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.38ms
[D 2023-11-16 20:07:54.040 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 1.18ms
[D 2023-11-16 20:07:56.035 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.38ms
[D 2023-11-16 20:07:58.038 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 1.05ms
[D 2023-11-16 20:07:58.903 JupyterHub proxy:880] Proxy: Fetching GET http://proxy-api:8001/api/routes
[D 2023-11-16 20:07:58.909 JupyterHub proxy:392] Checking routes
[D 2023-11-16 20:08:00.036 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.75ms
[D 2023-11-16 20:08:02.037 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 1.11ms
[D 2023-11-16 20:08:04.039 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.72ms
User-Pod:
[I 2023-11-16 20:07:40.998 JupyterHub log:191] 200 GET /hub/login (@::ffff:10.244.0.1) 0.96ms
[D 2023-11-16 20:07:42.036 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.36ms
[D 2023-11-16 20:07:43.312 JupyterHub roles:281] Assigning default role to User c
[I 2023-11-16 20:07:43.321 JupyterHub roles:238] Adding role user for User: c
[D 2023-11-16 20:07:43.329 JupyterHub roles:281] Assigning default role to User c
[D 2023-11-16 20:07:43.330 JupyterHub base:587] Setting cookie jupyterhub-session-id: {'httponly': True, 'path': '/'}
[D 2023-11-16 20:07:43.330 JupyterHub base:591] Setting cookie for c: jupyterhub-hub-login
[D 2023-11-16 20:07:43.330 JupyterHub base:587] Setting cookie jupyterhub-hub-login: {'httponly': True, 'path': '/hub/'}
[I 2023-11-16 20:07:43.330 JupyterHub base:837] User logged in: c
[D 2023-11-16 20:07:43.330 JupyterHub user:431] Creating <class 'kubespawner.spawner.KubeSpawner'> for c:
[I 2023-11-16 20:07:43.331 JupyterHub log:191] 302 POST /hub/login?next= -> /hub/spawn (c@::ffff:10.244.0.1) 21.55ms
[D 2023-11-16 20:07:43.338 JupyterHub base:299] Recording first activity for <User(c 0/1 running)>
[D 2023-11-16 20:07:43.341 JupyterHub scopes:877] Checking access to /hub/spawn via scope servers
[D 2023-11-16 20:07:43.341 JupyterHub scopes:690] Argument-based access to /hub/spawn via servers
[D 2023-11-16 20:07:43.346 JupyterHub pages:209] Serving options form for c
[I 2023-11-16 20:07:43.347 JupyterHub log:191] 200 GET /hub/spawn (c@::ffff:10.244.0.1) 9.53ms
[D 2023-11-16 20:07:44.037 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.76ms
[D 2023-11-16 20:07:44.242 JupyterHub scopes:877] Checking access to /hub/spawn via scope servers
[D 2023-11-16 20:07:44.242 JupyterHub scopes:690] Argument-based access to /hub/spawn via servers
[D 2023-11-16 20:07:44.242 JupyterHub pages:257] Triggering spawn with supplied form options for c
[D 2023-11-16 20:07:44.242 JupyterHub base:961] Initiating spawn for c
[D 2023-11-16 20:07:44.242 JupyterHub base:965] 1/64 concurrent spawns
[D 2023-11-16 20:07:44.242 JupyterHub base:970] 1 active servers
[I 2023-11-16 20:07:44.252 JupyterHub provider:659] Creating oauth client jupyterhub-user-c
[D 2023-11-16 20:07:44.259 JupyterHub user:794] Calling Spawner.start for c
[D 2023-11-16 20:07:44.259 JupyterHub spawner:3199] Applying KubeSpawner override for profile '<big>Tensorflow 2.10 ohne GPU Support</big>'
[D 2023-11-16 20:07:44.259 JupyterHub spawner:3177] .. overriding KubeSpawner value extra_labels={'hub.jupyter.org/network-access-hub': 'true'}
[D 2023-11-16 20:07:44.259 JupyterHub spawner:3177] .. overriding KubeSpawner value namespace=other-namespace
[I 2023-11-16 20:07:44.259 JupyterHub log:191] 302 POST /hub/spawn?_xsrf=[secret] -> /hub/spawn-pending/c?_xsrf=[secret] (c@::ffff:10.244.0.1) 18.95ms
[I 2023-11-16 20:07:44.261 JupyterHub spawner:2497] Attempting to create pod jupyter-c, with timeout 3
[D 2023-11-16 20:07:44.264 JupyterHub scopes:877] Checking access to /hub/spawn-pending/c via scope servers
[D 2023-11-16 20:07:44.264 JupyterHub scopes:690] Argument-based access to /hub/spawn-pending/c via servers
[I 2023-11-16 20:07:44.264 JupyterHub pages:398] c is pending spawn
[I 2023-11-16 20:07:44.265 JupyterHub log:191] 200 GET /hub/spawn-pending/c?_xsrf=[secret] (c@::ffff:10.244.0.1) 1.59ms
[D 2023-11-16 20:07:44.303 JupyterHub scopes:877] Checking access to /hub/api/users/c/server/progress via scope read:servers
[D 2023-11-16 20:07:44.304 JupyterHub scopes:690] Argument-based access to /hub/api/users/c/server/progress via read:servers
[D 2023-11-16 20:07:44.304 JupyterHub spawner:2296] progress generator: jupyter-c
[D 2023-11-16 20:07:46.036 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.34ms
[I 2023-11-16 20:07:46.428 JupyterHub log:191] 200 GET /hub/api (@10.244.0.13) 0.32ms
[D 2023-11-16 20:07:46.443 JupyterHub base:299] Recording first activity for <APIToken('ad22...', user='c', client_id='jupyterhub')>
[D 2023-11-16 20:07:46.447 JupyterHub scopes:877] Checking access to /hub/api/users/c/activity via scope users:activity
[D 2023-11-16 20:07:46.447 JupyterHub scopes:690] Argument-based access to /hub/api/users/c/activity via users:activity
[D 2023-11-16 20:07:46.448 JupyterHub users:874] Activity for user c: 2023-11-16T20:07:46.416655Z
[D 2023-11-16 20:07:46.448 JupyterHub users:892] Activity on server c/: 2023-11-16T20:07:46.416655Z
[I 2023-11-16 20:07:46.450 JupyterHub log:191] 200 POST /hub/api/users/c/activity (c@10.244.0.13) 7.33ms
[D 2023-11-16 20:07:48.037 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.91ms
[D 2023-11-16 20:07:50.039 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.87ms
[D 2023-11-16 20:07:52.036 JupyterHub log:191] 200 GET /hub/health (@10.244.0.1) 0.38ms
Do you have an idea what else I can try?