403 Forbidden: Authorization form must be sent from authorization page

Problem

I’m trying to setup BinderHub using Keycloak authentication, the authentication with Keycloak is succesful but I can’t seem to get the authorization right. I feel that I’m very close but I can’t get past the authorization phase because the referer url is in https, while the request url is in http. I have tried numerous things in the config.yaml, some examples:

  • Change everything in the config.yaml to http
  • Change everything in the config.yaml to https
  • Countless tries where some are in http and others are not
  • Mess with JupyterHub proxy settings
  • Try out things with Ingress
  • Tried suggestions from here: https://github.com/jupyterhub/jupyterhub/issues/2284

Hoping someone can assist me. :slightly_smiling_face:

Setup

A SSL termination proxy that:

A bare-metal Kubernetes where BinderHub is deployed using the config.yaml that can be found below.

Authorization phase

After I click on “Authorize” I end up with a 403.


JupyterHub logs from authorization phase

Config.yaml

---
# External SSL termination proxy points to NodePort
service:
  type: NodePort
config: 
  BinderHub:
    # Not sure if this should be http or https. I have tried both multiple times.
    hub_url: https://jhub.xxx.xxx.io
    image_prefix: gcr.io/xxx/binder-dev
    use_registry: true
    debug: true
    auth_enabled: true
resources:
  requests:
    cpu: 2
    memory: 1024Mi
imageCleaner:
  enabled: false
dind:
  enabled: true
  daemonset:
    image:
      name: docker
      tag: 18.09.2-dind
jupyterhub:
  cull:
    # don't cull authenticated users
    users: False
  hub:
    extraEnv:
      OAUTH2_AUTHORIZE_URL: https://auth.xxx.xxx.io/auth/realms/xxx/protocol/openid-connect/auth
      OAUTH2_TOKEN_URL: https://auth.xxx.xxx.io/auth/realms/xxx/protocol/openid-connect/token
      OAUTH_CALLBACK_URL: https://jhub.xxx.xxx.io/hub/oauth_callback
    services:
      binder:
        # Not sure if this should be http or https. I have tried both multiple times.
        oauth_redirect_uri: "https://binderhub.xxx.xxx.io/oauth_callback"
        oauth_client_id: "keycloak-xxx-client"
    extraConfig:
      hub_extra: |
        c.JupyterHub.redirect_to_server = False

      binder: |
        from kubespawner import KubeSpawner

        class BinderSpawner(KubeSpawner):
          def start(self):
              if 'image' in self.user_options:
                # binder service sets the image spec via user options
                self.image = self.user_options['image']
              return super().start()
        c.JupyterHub.spawner_class = BinderSpawner
    db: 
      pvc: 
        storageClassName: rook-ceph-block
  singleuser:
    storage: 
      dynamic: 
        storageClass: rook-ceph-block
      type: dynamic
  auth:
    type: custom
    custom:
      className: oauthenticator.generic.GenericOAuthenticator
      config:
        login_service: "keycloak"
        client_id: "keycloak-xxx-client"
        client_secret: "xxx-xxx-xxx-xxx"
        token_url: https://auth.xxx.xxx.io/auth/realms/xxx/protocol/openid-connect/token
        userdata_url: https://auth.xxx.xxx.io/auth/realms/xxx/protocol/openid-connect/userinfo
        userdata_method: GET
        userdata_params: {'state': 'state'}
        username_key: preferred_username
  proxy:
    service:
      type: NodePort

After following this suggestion I was able to authorize and access BinderHub but then I stumbled onto a new problem and I have a feeling it still has something to do with http headers.

When I launch a binder the server is created successfully but when I then try to access my jupyter server I get a 404. JupyterHub cannot seem to find the static files:

I found the solution, it had nothing to do with the headers. I forgot to add this line to the config.yaml:

jupyterhub:
  singleuser:
    # to make notebook servers aware of hub
    cmd: jupyterhub-singleuser`

This was mentioned in the authentication documentation: https://binderhub.readthedocs.io/en/latest/authentication.html