Binderhub + Jupyterhub 4 - Error creating user xxxxxx: HTTP 403: Forbidden

Hi :wave:,

I am trying to run a single instance BinderHub based on the scripts provided in binderhub/testing/local-binder-local-hub at main 路 jupyterhub/binderhub 路 GitHub. I鈥檝e successfully setup single instance hubs previously without an issue like this.

In setting up another instance of a binderhub, I am now hitting an error using during the launch of the user server - (everything builds ok).

I see the following in the server logs:

Error creating user curvenote-binder-base-c1rhebyn: HTTP 403: Forbidden
    b'{"status": 403, "message": "Action is not authorized with current scopes; requires any of [admin:users]"}'
[E 230519 15:53:24 builder:691] Retrying launch of https://github.com/curvenote/binder-base after error (duration=0s, attempt=1): HTTPError()
[I 230519 15:53:28 launcher:197] Creating user curvenote-binder-base-jmxbpyz6 for image curvenote-2dbinder-2dbase-fc751c:516048bd6b2577dc8ae4c35d0c8f7f51e6839153
[E 230519 15:53:28 launcher:207] Error creating user curvenote-binder-base-jmxbpyz6: HTTP 403: Forbidden

After trying to debug and also comparing to one of my working deployments I can see that this latest deployment is using jupyterhub 4.0 while other working deployments are using jupyterhub 3.x, I鈥檓 assuming that this is the issue (perhaps new/improved permissions management in v4?) - maybe incorrectly, but assuming that it is, I am trying to modify the configuration to add the scopes to the appropriate service.

I鈥檝e added the load_roles to my jupyterhub_config.py, here鈥檚 a relevant section including what I added. I鈥檓 not clear if this is the right way to go or whether it is indeed the binder service that is trying to create that user.

...
c.JupyterHub.services = [
    {
        "name": binderhub_service_name,
        "admin": True,
        "command": ["python3", "-mbinderhub", f"--config={binderhub_config}"],
        "url": "http://localhost:8585",
        "environment": {
            "JUPYTERHUB_EXTERNAL_URL": os.getenv("JUPYTERHUB_EXTERNAL_URL", "")
        },
    }
]
c.JupyterHub.default_url = f"/services/{binderhub_service_name}/"
c.JupyterHub.load_roles = [{
	"name": "binder",
	"services": [binderhub_service_name], 
	"scopes": ["access:services", "admin:users", "admin:servers"]
}]

PS I鈥檓 also struggling a bit to work across the docs, as it鈥檚 not always clear how to relate settings in a config.yml to the binderhub_config.py and juptyerhub_config.py

I鈥檓 about to downgrade to jupyterhub 3.1.1 to see if this resolves this, but i would be great to get this working on 4 (if this is the issue), and I鈥檇 be happy to open a PR with updated testing scripts if I can get is working.

To summarize my key questions:

  • is this a jupyterhub 4 related issue?
  • how can I assign the scopes properly in jupyterhub_config.py or otherwise add to the configuration to deal with that?

Any help / pointers appreciated!
Steve

ok just reverting jupyterhub to 3.1.1 did not resolve this, differences between my other deployments aside, I鈥檓 not sure what else can be causing this

figuring out that

    loadRoles:
      user:
        scopes:
          - self
          - "access:services"

as per the docs here

means

c.JupyterHub.load_roles = [
	{
		"name": "user",
		"description": "Allow users to access binder service",
		"scopes": ["self", "access:services"]
	}
]

in a jupyterhub_config.py file,

2 Likes