Binderhub + Jupyterhub 4 - Error creating user xxxxxx: HTTP 403: Forbidden

Hi :wave:,

I am trying to run a single instance BinderHub based on the scripts provided in binderhub/testing/local-binder-local-hub at main 路 jupyterhub/binderhub 路 GitHub. I鈥檝e successfully setup single instance hubs previously without an issue like this.

In setting up another instance of a binderhub, I am now hitting an error using during the launch of the user server - (everything builds ok).

I see the following in the server logs:

Error creating user curvenote-binder-base-c1rhebyn: HTTP 403: Forbidden
    b'{"status": 403, "message": "Action is not authorized with current scopes; requires any of [admin:users]"}'
[E 230519 15:53:24 builder:691] Retrying launch of after error (duration=0s, attempt=1): HTTPError()
[I 230519 15:53:28 launcher:197] Creating user curvenote-binder-base-jmxbpyz6 for image curvenote-2dbinder-2dbase-fc751c:516048bd6b2577dc8ae4c35d0c8f7f51e6839153
[E 230519 15:53:28 launcher:207] Error creating user curvenote-binder-base-jmxbpyz6: HTTP 403: Forbidden

After trying to debug and also comparing to one of my working deployments I can see that this latest deployment is using jupyterhub 4.0 while other working deployments are using jupyterhub 3.x, I鈥檓 assuming that this is the issue (perhaps new/improved permissions management in v4?) - maybe incorrectly, but assuming that it is, I am trying to modify the configuration to add the scopes to the appropriate service.

I鈥檝e added the load_roles to my, here鈥檚 a relevant section including what I added. I鈥檓 not clear if this is the right way to go or whether it is indeed the binder service that is trying to create that user.

... = [
        "name": binderhub_service_name,
        "admin": True,
        "command": ["python3", "-mbinderhub", f"--config={binderhub_config}"],
        "url": "http://localhost:8585",
        "environment": {
c.JupyterHub.default_url = f"/services/{binderhub_service_name}/"
c.JupyterHub.load_roles = [{
	"name": "binder",
	"services": [binderhub_service_name], 
	"scopes": ["access:services", "admin:users", "admin:servers"]

PS I鈥檓 also struggling a bit to work across the docs, as it鈥檚 not always clear how to relate settings in a config.yml to the and

I鈥檓 about to downgrade to jupyterhub 3.1.1 to see if this resolves this, but i would be great to get this working on 4 (if this is the issue), and I鈥檇 be happy to open a PR with updated testing scripts if I can get is working.

To summarize my key questions:

  • is this a jupyterhub 4 related issue?
  • how can I assign the scopes properly in or otherwise add to the configuration to deal with that?

Any help / pointers appreciated!

ok just reverting jupyterhub to 3.1.1 did not resolve this, differences between my other deployments aside, I鈥檓 not sure what else can be causing this

figuring out that

          - self
          - "access:services"

as per the docs here


c.JupyterHub.load_roles = [
		"name": "user",
		"description": "Allow users to access binder service",
		"scopes": ["self", "access:services"]

in a file,