Hello,
I have setup Generic Oathu for hub. Its working fine. SSO sigin happening well. But i have noticed a thing.
Please let us know if you need more information
/user/* should be protected by the JupyterHub login.
Are you sure the incognito window created a completely new session? I’ve found that occasionally an incognito window can retain some old state, which means you’re still logged in to the OAuth provider.
Can you show us your full browser console network requests?
Can you show me the full set of network requests, from the very start?
Can you also turn on debug logging and show us your hub logs? It might also be helpful to see your Z2JH configuration.
[D 2024-04-15 12:40:14.545 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.82ms
[W 2024-04-15 12:40:14.733 JupyterHub base:415] Invalid or expired cookie token
[I 2024-04-15 12:40:14.734 JupyterHub log:191] 302 GET /hub/ → /hub/login?next=%2Fhub%2F (@192.168.0.218) 1.87ms
[I 2024-04-15 12:40:15.132 JupyterHub log:191] 302 GET /hub/login?next=%2Fhub%2F → /hub/oauth_login?next=%2Fhub%2F (@192.168.0.218) 2.01ms
[I 2024-04-15 12:40:15.522 JupyterHub oauth2:97] OAuth redirect: https://gateway.dev.dna.app.corpintra.net/hub/oauth_callback
[D 2024-04-15 12:40:15.522 JupyterHub base:587] Setting cookie oauthenticator-state: {‘httponly’: True, ‘expires_days’: 1}
[I 2024-04-15 12:40:15.523 JupyterHub log:191] 302 GET /hub/oauth_login?next=%2Fhub%2F → https://sso-int.mercedes-benz.com/as/authorization.oauth2?response_type=code&redirect_uri=https%3A%2F%2Fgateway.dev.dna.app.corpintra.net%2Fhub%2Foauth_callback&client_id=ccd68299-0bc6-4e45-8ea1-11dd1a31d2a7&state=[secret]&scope=openid+email+group_type (@192.168.0.218) 1.73ms
[D 2024-04-15 12:40:16.545 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.67ms
[D 2024-04-15 12:40:18.544 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.67ms
[D 2024-04-15 12:40:18.701 JupyterHub roles:281] Assigning default role to User palansa
[D 2024-04-15 12:40:18.705 JupyterHub base:591] Setting cookie for palansa: jupyterhub-hub-login
[D 2024-04-15 12:40:18.705 JupyterHub base:587] Setting cookie jupyterhub-hub-login: {‘httponly’: True, ‘path’: ‘/hub/’}
[I 2024-04-15 12:40:18.705 JupyterHub base:837] User logged in: palansa
[I 2024-04-15 12:40:18.706 JupyterHub log:191] 302 GET /hub/oauth_callback?code=[secret]&state=[secret] → /hub/ (@192.168.0.218) 69.98ms
[I 2024-04-15 12:40:19.081 JupyterHub log:191] 302 GET /hub/ → /hub/home (palansa@192.168.0.218) 17.83ms
[D 2024-04-15 12:40:19.422 JupyterHub user:431] Creating <class ‘CodeSpaceKubeSpawner’> for palansa:
[D 2024-04-15 12:40:35.740 JupyterHub scopes:877] Checking access to /hub/spawn/palansa/ws1383 via scope servers
[D 2024-04-15 12:40:35.740 JupyterHub scopes:677] Unrestricted access to /hub/spawn/palansa/ws1383 via servers
[D 2024-04-15 12:40:35.740 JupyterHub user:431] Creating <class ‘CodeSpaceKubeSpawner’> for palansa:ws1383
[D 2024-04-15 12:40:35.752 JupyterHub pages:209] Serving options form for palansa:ws1383
[I 2024-04-15 12:40:35.757 JupyterHub log:191] 200 GET /hub/spawn/palansa/ws1383 (palansa@192.168.0.218) 19.60ms
[D 2024-04-15 12:40:36.544 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.97ms
[D 2024-04-15 12:40:37.661 JupyterHub scopes:877] Checking access to /hub/spawn/palansa/ws1383 via scope servers
[D 2024-04-15 12:40:37.661 JupyterHub scopes:677] Unrestricted access to /hub/spawn/palansa/ws1383 via servers
[D 2024-04-15 12:40:37.661 JupyterHub pages:257] Triggering spawn with supplied form options for palansa:ws1383
[D 2024-04-15 12:40:37.661 JupyterHub base:961] Initiating spawn for palansa:ws1383
[D 2024-04-15 12:40:37.661 JupyterHub base:965] 0/64 concurrent spawns
[D 2024-04-15 12:40:37.662 JupyterHub base:970] 0 active servers
[I 2024-04-15 12:40:37.700 JupyterHub provider:659] Creating oauth client jupyterhub-user-palansa-ws1383
[D 2024-04-15 12:40:37.740 JupyterHub user:794] Calling Spawner.start for palansa:ws1383
[D 2024-04-15 12:40:37.741 JupyterHub spawner:3220] Applying KubeSpawner override for profile ‘Default environment’
[D 2024-04-15 12:40:37.741 JupyterHub spawner:3198] … overriding KubeSpawner value http_timeout=180
[D 2024-04-15 12:40:37.742 JupyterHub spawner:3198] … overriding KubeSpawner value image=registry-emea.app.corpintra.net/dnaplatform/cs-generic:4.22.1
[D 2024-04-15 12:40:37.742 JupyterHub spawner:3198] … overriding KubeSpawner value start_timeout=180
[I 2024-04-15 12:40:37.750 JupyterHub log:191] 302 POST /hub/spawn/palansa/ws1383?_xsrf=[secret] → /hub/spawn-pending/palansa/ws1383?_xsrf=[secret] (palansa@192.168.0.218) 92.09ms
[I 2024-04-15 12:40:37.773 JupyterHub reflector:282] watching for pods with label selector=‘component=singleuser-server’ in namespace code-server
[D 2024-04-15 12:40:37.773 JupyterHub reflector:289] Connecting pods watcher
[I 2024-04-15 12:40:37.928 JupyterHub reflector:282] watching for events with field selector=‘involvedObject.kind=Pod’ in namespace code-server
[D 2024-04-15 12:40:37.928 JupyterHub reflector:289] Connecting events watcher
[I 2024-04-15 12:40:37.935 JupyterHub spawner:2551] Attempting to create pvc ws1383, with timeout 3
[I 2024-04-15 12:40:38.017 JupyterHub spawner:2567] PVC ws1383 already exists, so did not create new pvc.
[D 2024-04-15 12:40:38.019 JupyterHub scopes:877] Checking access to /hub/spawn-pending/palansa/ws1383 via scope servers
[D 2024-04-15 12:40:38.019 JupyterHub scopes:677] Unrestricted access to /hub/spawn-pending/palansa/ws1383 via servers
[I 2024-04-15 12:40:38.020 JupyterHub pages:398] palansa:ws1383 is pending spawn
[I 2024-04-15 12:40:38.025 JupyterHub log:191] 200 GET /hub/spawn-pending/palansa/ws1383?_xsrf=[secret] (palansa@192.168.0.218) 8.65ms
[I 2024-04-15 12:40:38.035 JupyterHub spawner:2511] Attempting to create pod ws1383, with timeout 3
[D 2024-04-15 12:40:38.408 JupyterHub scopes:877] Checking access to /hub/api/users/palansa/servers/ws1383/progress via scope read:servers
[D 2024-04-15 12:40:38.408 JupyterHub scopes:677] Unrestricted access to /hub/api/users/palansa/servers/ws1383/progress via read:servers
[D 2024-04-15 12:40:38.410 JupyterHub spawner:2303] progress generator: ws1383
[D 2024-04-15 12:40:38.545 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 1.02ms
[D 2024-04-15 12:40:40.545 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 1.05ms
[D 2024-04-15 12:40:42.544 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.69ms
[D 2024-04-15 12:40:42.731 JupyterHub proxy:880] Proxy: Fetching GET http://cs-hub-proxy-api:8001/api/routes
[D 2024-04-15 12:40:42.737 JupyterHub proxy:392] Checking routes
[D 2024-04-15 12:40:44.545 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.72ms
[D 2024-04-15 12:40:46.544 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.64ms
[D 2024-04-15 12:40:47.798 JupyterHub reflector:374] pods watcher timeout
[D 2024-04-15 12:40:47.798 JupyterHub reflector:289] Connecting pods watcher
[W 2024-04-15 12:40:47.979 JupyterHub web:1869] 403 GET /hub/metrics (192.168.130.55): Access to metrics requires scope ‘read:metrics’
[D 2024-04-15 12:40:47.980 JupyterHub base:1371] No template for 403
[W 2024-04-15 12:40:47.980 JupyterHub log:191] 403 GET /hub/metrics (@192.168.130.55) 3.59ms
[D 2024-04-15 12:40:48.021 JupyterHub reflector:374] events watcher timeout
[D 2024-04-15 12:40:48.021 JupyterHub reflector:289] Connecting events watcher
[D 2024-04-15 12:40:48.544 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.91ms
[D 2024-04-15 12:40:49.321 JupyterHub spawner:2821] pod code-server/ws1383 events before launch: 2024-04-15T12:40:38.088130Z [Normal] Successfully assigned code-server/ws1383 to c52p212-md-68f7b595ff-g6qlr
2024-04-15T12:40:40Z [Normal] AttachVolume.Attach succeeded for volume “pvc-de78a918-797b-47f3-9745-4b40fb376eb2”
2024-04-15T12:40:46Z [Normal] Container image “registry-emea.app.corpintra.net/dnaplatform/cs-generic:4.22.1” already present on machine
2024-04-15T12:40:46Z [Normal] Created container copy-pkg-install-sh
2024-04-15T12:40:46Z [Normal] Started container copy-pkg-install-sh
2024-04-15T12:40:48Z [Normal] Container image “registry-emea.app.corpintra.net/dnaplatform/cs-generic:4.22.1” already present on machine
2024-04-15T12:40:48Z [Normal] Created container notebook
2024-04-15T12:40:48Z [Normal] Started container notebook
[D 2024-04-15 12:40:49.340 JupyterHub spawner:1384] Polling subprocess every 30s
[D 2024-04-15 12:40:50.544 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.71ms
[D 2024-04-15 12:40:51.594 JupyterHub utils:278] Server at http://192.168.130.114:8888/user/palansa/ws1383/ responded with 404
[W 2024-04-15 12:40:51.594 JupyterHub _version:37] Single-user server has no version header, which means it is likely < 0.8. Expected 4.0.2
[I 2024-04-15 12:40:51.594 JupyterHub base:990] User palansa:ws1383 took 13.933 seconds to start
[I 2024-04-15 12:40:51.594 JupyterHub proxy:330] Adding user palansa to proxy /user/palansa/ws1383/ => http://192.168.130.114:8888
[D 2024-04-15 12:40:51.595 JupyterHub proxy:880] Proxy: Fetching POST http://cs-hub-proxy-api:8001/api/routes/user/palansa/ws1383
[I 2024-04-15 12:40:51.622 JupyterHub users:768] Server palansa:ws1383 is ready
[I 2024-04-15 12:40:51.623 JupyterHub log:191] 200 GET /hub/api/users/palansa/servers/ws1383/progress?_xsrf=[secret] (palansa@192.168.0.218) 13217.51ms
[D 2024-04-15 12:40:52.022 JupyterHub scopes:877] Checking access to /hub/spawn-pending/palansa/ws1383 via scope servers
[D 2024-04-15 12:40:52.022 JupyterHub scopes:677] Unrestricted access to /hub/spawn-pending/palansa/ws1383 via servers
[I 2024-04-15 12:40:52.023 JupyterHub log:191] 302 GET /hub/spawn-pending/palansa/ws1383?_xsrf=[secret] → /user/palansa/ws1383/ (palansa@192.168.0.218) 3.51ms
[D 2024-04-15 12:40:52.544 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.79ms
[D 2024-04-15 12:40:54.545 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 0.66ms
[D 2024-04-15 12:40:56.546 JupyterHub log:191] 200 GET /hub/health (@192.168.0.140) 1.41ms
[D 2024-04-15 12:40:57.819 JupyterHub reflector:374] pods watcher timeout
[D 2024-04-15 12:40:57.819 JupyterHub reflector:289] Connecting pods watcher
[D 2024-04-15 12:40:58.194 JupyterHub reflector:374] events watcher timeout
[D 2024-04-15 12:40:58.195 JupyterHub reflector:289] Connecting events watcher
[D 2024-04-15 12:40:58.544 JupyterHub log:191] 200 GET /hub/health (@192.
In your hub logs can you indicate (e.g. by timestamp ranges) which of those logs correspond to the original login and starting of the server, and which correspond to the requests from the incognito window that shouldn’t be allowed?
Can you also show the full set of requests for the incognito browser, starting from when you first open the incognito window?
Your screenshot looks like it’s running vs code. Can you share what the launch command inside the container is (it should be something lke jupyterhub-singleuser) and the logs from the pod?
Blockquote
extraConfig:
code_spawner.py: |
from kubespawner.spawner import KubeSpawner
from jupyterhub.spawner import _quote_safe
class CodeSpaceKubeSpawner(KubeSpawner):
def get_args(self):
"""Custom args function for the coder"""
# Turn off authentication (happens via jupyterhub)
args = ["--auth", "none"]
# Turn off telemetry
args += ["--disable-telemetry"]
# Set App Name
args += ["--app-name", "DnACodeSpace"]
# set port and ip if given
ip = "0.0.0.0"
if self.ip:
ip = _quote_safe(self.ip)
port = 8888
if self.port:
port = self.port
elif self.server and self.server.port:
self.log.warning(
"Setting port from user.server is deprecated as of JupyterHub 0.7."
)
port = self.server.port
args += ["--bind-addr", f"{ip}:{port}"]
# set startup folder
if self.notebook_dir:
notebook_dir = self.format_string(self.notebook_dir)
args += ["--user-data-dir", _quote_safe(notebook_dir)]
# if self.debug:
# args += ["-vvv"]
args.extend(self.args)
return args
def get_env(self):
env = super().get_env()
user_options_env = self.user_options.get("env", {})
env.update(user_options_env)
return env
# Use the configured spawner
c.JupyterHub.spawner_class = CodeSpaceKubeSpawner
code_settings.py: |
# The working dir is by default set to
# /home/coder in the VSCode image
c.CodeSpaceKubeSpawner.working_dir = "/home/coder"
# By default, the cmd includes the call to "jupyterhub-singleserver"
# However, the docker image already comes with the correct
# VSCode command to call, so we just set it to an empty string here
c.CodeSpaceKubeSpawner.cmd = ""
# Dynamically Get the set_resource capacity from the user options or set default values
def set_resource(spawner):
# user_storage_capacity = spawner.storage_capacity
# user_pvc_name = spawner.pvc_name
spawner.storage_capacity = spawner.user_options.get("storage_capacity", "1Gi")
spawner.mem_guarantee = spawner.user_options.get("mem_guarantee", "256M")
spawner.mem_limit = spawner.user_options.get("mem_limit", "4G")
spawner.cpu_limit = spawner.user_options.get("cpu_limit", 3)
spawner.cpu_guarantee = spawner.user_options.get("cpu_guarantee", 0.2)
# expand_pvc_command = f'kubectl patch pvc {user_pvc_name} -p \'{{"spec":{{"resources":{{"requests":{{"storage":"{user_storage_capacity}"}}}}}}}}\''
# os.system(expand_pvc_command)
c.CodeSpaceKubeSpawner.pre_spawn_hook = set_resource
c.CodeSpaceKubeSpawner.extra_container_config = {
'livenessProbe': {
'httpGet': {
'path': '/healthz',
'port': 'notebook-port',
'scheme': 'HTTP'
},
'initialDelaySeconds': 30,
'periodSeconds': 60,
'timeoutSeconds': 5,
'successThreshold': 1,
'failureThreshold': 5
}
}
code_settings.py: |
//comments
The working dir is by default set to
/home/coder in the VSCode image
c.CodeSpaceKubeSpawner.working_dir = “/home/coder”
By default, the cmd includes the call to “jupyterhub-singleserver”
However, the docker image already comes with the correct
VSCode command to call, so we just set it to an empty string here
//end of commands
c.CodeSpaceKubeSpawner.cmd = “”
I believe this is disabling auth altogether. This is not an argument understood by jupyterhub-singleuser, which means your launch command is presumably launching vscode directly without any auth enabled.
The singleuser command is responsible for implementing authentication, so if you launch a command with no auth enabled, there will be no auth protecting your servers.
If you want to rely in the default jupyterhub authentication, you can put your command behind jupyter-server-proxy.
Alternatively, if your application supports OAuth, you can use jupyterhub as an oauth provider.
@minrk thanks a lot i will have look these docs.
Yes you are right i’m lunching the vscode directly. auth --none is the cmd for vs code im passing for startup
As @minrk suggested, using jupyter-server-proxy is a better way to do it. Both code-server and jupyter-server-proxy supports unix sockets so that you can disabled password based auth and use unix sockets to securely provide VSCode to your users. Here is an example of entry point to jupyter-server-proxy for code-server