Help setting up Binderhub on GKE on existing ingress

I am trying to setup Binderhub on a GKE cluster behind an existing ingress. I want to be able to access Binderhub at website/binder/.

My config.yaml:

service:
  type: ClusterIP

jupyterhub:
  proxy:
    service:
      type: ClusterIP
  hub:
    baseUrl: jupyterhub

config:
  BinderHub: 
    debug: true
    use_registry: true
    base_url: /binder/
    image_prefix: us-central1-docker.pkg.dev/myproject/binderhub/binder
    hub_url: http://website.com/jupyterhub

my ingress config:

- pathType: Prefix
  path: "/binder"
  backend:
      service:
        name: binderhub-proxy-svc
        port:
            number: 80

- pathType: Prefix
  path: "/jupyterhub"
  backend:
      service:
        name: binderhub-proxy-svc
        port:
            number: 80

my proxy configuration:

    location /binder {
        proxy_pass http://binder:80/binder;
        proxy_set_header Host $host;
        proxy_redirect off;
        auth_basic "Please authenticate!";
        auth_basic_user_file /etc/nginx/.htpasswd;
    }

    location /jupyterhub {
        proxy_pass http://proxy-public:80/jupyterhub;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

I can successfully access website.com/binder/ and website.com/jupyterhub/hub/ (which redirects to jupyterhub/hub/login?next=%2Fjupyterhub%2Fhub%2F where I see a 403 message), however when I try to launch a repository I see a failure.

Looking at the logs of the hub pod:

[W 2025-03-04 09:55:14.690 JupyterHub web:1873] 403 GET /jupyterhub/hub/api/users/yunabe-lgo-binder-847rx0p4 (::ffff:10.92.0.42): Missing or invalid credentials.

The logs of the binder pod:

[E 250304 09:58:20 launcher:208] Error creating user yunabe-lgo-binder-847rx0p4: HTTP 403: Forbidden
    b'{"status": 403, "message": "Missing or invalid credentials."}'
[E 250304 09:58:20 builder:749] Retrying launch of https://github.com/yunabe/lgo-binder after error (duration=0s, attempt=3): HTTPError()

I tried setting allow_all but nothing changed:

jupyterhub:
  proxy:
    service:
      type: ClusterIP
  hub:
    baseUrl: /jupyterhub/
    config:
      Authenticator:
        allow_all: true

If I use DummyAuthenticator then I can login with any user, but the launcher still fails. How do I tell instruct the launcher to use this password?

jupyterhub:
  proxy:
    service:
      type: ClusterIP
  hub:
    baseUrl: /jupyterhub/
    config:
      Authenticator:
        allow_all: True
      DummyAuthenticator:
        password: foobar
      JupyterHub:
        authenticator_class: dummy

looks like maybe I need to create a token manually and explicitly add it to config.yaml?

config:
  BinderHub: 
    debug: true
    use_registry: true
    base_url: /binder/
    image_prefix: us-central1-docker.pkg.dev/myproject/binderhub/binder
    hub_url: http://website.com/jupyterhub
    hub_api_token: token123...

tokens don’t work either

This is currently being answered on Jupyterhub 403 forbidden even with allow_all: true