Shared links with access tokens still result in 403s

Hi there!

I have a Z2JH JupyterHub installation set up with a custom OAuthenticator that just implements OIDC (with AWS Cognito).

I’m running the latest Z2JH with JupyterHub 4.0.2 and JupyterLab 4.0.8.

When a user generates a shared link with a token (either using jupyterlab-link-share or the new RTC functionality builtin to JupyterLab - jupyter-collaboration) the receiving user is able to access the shared server (yay!) but every subsequent API request yields a 403 (boo) and refreshing the page also yields a total 403 page (boo). Essentially the page “loads” but everything on it is broken.

I’m not sure where to even start here. The token that gets generated “works” but then disappears after the first page load. Where has it gone? Is it being replaced with the receiving user’s existing session, which otherwise does not have permission to access the shared server?

Well it isn’t falling back to the active user session. I disabled auto login, signed out of JupyterHub, and hit a share link with an included token. Same behavior.

This time, the target singleuser server logs indicate:

No user identified
Couldn’t authenticate WebSocket connection

Whereas before it simply indicated that the active user did not have the appropriate scopes.

I saw this post which looks similar (at least we’re both running into Couldn't authenticate WebSocket connection).

I tried setting

c.Spawner.oauth_client_allowed_scopes = ['inherit']

And did not observe any difference in behavior :frowning:

I’m not entirely clear what authentication state is or does, but I’m grasping at straws so I enabled on that on my JupyterHub installation as well with no success.

Just ran across this thread which seems related.

Although @minrk does not provide a specific issue link for the bug they’ve discovered, I set the relevant environment variable:

c.Spawner.environment = {

and it DOES seem to have fixed the websocket 403 issue. I don’t understand why or what changed, but this is a workable solution for the time being.

Adding some context here,

The bugfix was here, sorry for not linking earlier.

It is indeed about persisting the token in the URL, which the extension didn’t preserve from the original behavior (the extension is meant to be a reimplementation with no changes in behavior, so any chagne is a bug, which is also why we will have the JUPTYERHUB_SINGLEUSER_EXTENSION=0 fallback at least until we are confident all of these differences are worked out).

The key points to get are:

  • single-user servers use the Hub as an OAuth provider
  • requests to single-user servers are authenticated with OAuth tokens in one of:
    1. Authorization header
    2. (encrypted) cookie
    3. ?token= url parameter

When a request to a single-user server is made with the ?token= url parameter, that token is meant to be persisted in the cookie for future requests (so a URL constitutes a link to start a fully logged-in session). This is a Jupyter Server feature, not specific to JupyterHub, but JupyterHub inherits it. jupyterlab-link-share creates these links, and relies on this url->cookie persistence to function for requests after the first.

As an implementation detail, the way JupyterHub configures Jupyter Server for authentication with the Hub overrides this logic, and the Extension implementation omitted the “if a page request is authenticated with a token in the URL, persist it in a cookie” bit, which was fixed by this PR.

More details on jupyterhub’s use of oauth in the docs.

Also related, jupyterlab-link-share when used with JupyterHub is specifically enabling spoofing of users (i.e. you are creating a link that allows a user to become indistinguishable from another user via the link). A generally preferable way to share access to servers is via granting the access:servers or access:servers!server=user/ permission. Then link-share doesn’t do anything, since the user actually has (admin-granted) permission and will go through standard oauth to access the server, and regular links just work, and requests will continue to be identified as coming from the correct authenticated user.

@minrk thank you so much for the clarifying explanation and link to the bugfix! It looks like that hasn’t been released yet (4.0.2 does not contain the patch) so I will monitor the release feed with bated breath :slight_smile:

A generally preferable way to share access to servers is via granting the access:servers or access:servers!server=user/ permission.

I’ve seen this recommendation throughout the docs and I strongly agree with it myself – however, something I have been struggling with is the lack of a mechanism to grant these permissions on-the-fly.

The JupyterLab link sharing functionality (either builtin in 4.x or via jupyterlab-link-share) allows users to consent to another user’s access to their server, even if that access is implemented via spoofing (and even if tokens don’t expire automatically :worried:). This opt-in behavior is highly desirable. Think of it like adding a user to your Google Doc: while you’ve granted consent, they can access it.

As far as I can tell, permission grants are statically defined using role assignment (JupyterHub.load_roles). I’ve managed to make group membership dynamic by adding custom logic to OAuthenticator, but even then it would take an administrator to manipulate group membership just to allow users to share their servers.

Unless I’m missing some functionality in JupyterHub to dynamically grant access:servers (or other permissions) I fear I’m stuck with link sharing and user spoofing for the time being.

That’s totally understandable, and literally the one feature I want to land for 5.0 with this very-much-WIP PR. If you want to provide some details of your use case and desires there, that would help us make sure the first version is useful.

I knew I couldn’t be the only person in this boat – thank you so much! I will take my commentary to that PR.