Kecloak authentication with jupyterhub failed - 500 internal server error

hello everyone,

sorry that i reopened this topic.
Im facing the same issue running. here is the different informations:

Helm release name: jupyter
Helm chart version: 3.3.7
JupyterHub version: 4.1.5
Kubernetes v1.25

I configured jupyterhub with Keycloak as Login mechanism but i logged in but i received " 500 : Internal Server Error".

hub:
  config:
    JupyterHub:
      authenticator_class: generic-oauth
    Authenticator:
      enable_auth_state: true
    GenericOAuthenticator:
      client_id: "my-id"
      client_secret: "client-secret"
      oauth_callback_url: https://test.domain/jupyter/hub/oauth_callback
      authorize_url: https://test.domain/auth/realms/ui/protocol/openid-connect/auth
      token_url: https://test.domain/auth/realms/ui/protocol/openid-connect/token
      userdata_url: https://test.domain/auth/realms/ui/protocol/openid-connect/userinfo
      login_service: keycloak
      username_claim: email
      tls_verify: false
      userdata_params:
        state: state
  baseUrl: /jupyter
  cookieSecret: 93802138404847024044070447h447204704248072047h02470724

The log of hub pod looks like this

[I 2024-06-20 10:57:23.799 JupyterHub log:192] 302 GET /jupyter/hub/ -> /jupyter/hub/login?next=%2Fjupyter%2Fhub%2F (@173.10.0.15) 0.65ms
[I 2024-06-20 10:57:23.825 JupyterHub _xsrf_utils:125] Setting new xsrf cookie for b'None:gvavpxLj7KQj0O0NHZfbe7ROQdzTphE0l-DpQwsGAQo=' {'path': '/jupyter/hub/', 'max_age': 3600}
[I 2024-06-20 10:57:23.848 JupyterHub log:192] 200 GET /jupyter/hub/login?next=%2Fjupyter%2Fhub%2F (@173.10.0.15) 23.20ms
[I 2024-06-20 10:58:31.810 JupyterHub oauth2:99] OAuth redirect: https://test.domain/jupyter/hub/oauth_callback
[I 2024-06-20 10:58:31.811 JupyterHub log:192] 302 GET /jupyter/hub/oauth_login?next=%2Fjupyter%2Fhub%2F -> https://test.domain/auth/realms/my-id/protocol/openid-connect/auth?response_type=code&redirect_uri=https%3A%2F%2Ftest.domain%2Fjupyter%2Fhub%2Foauth_callback&client_id=my-id&state=[secret] (@173.10.0.15) 1.09ms
[E 2024-06-20 10:59:04.219 JupyterHub oauth2:653] Error Fetching user info... 403 GET https://test.domain/auth/realms/my-id/protocol/openid-connect/userinfo: 
[E 2024-06-20 10:59:04.219 JupyterHub web:1875] Uncaught exception GET /jupyter/hub/oauth_callback?state=eyJzdGF0ZV9pZCI6ICI0ZTEyMzM3MTkxOWY0OGI3YTk2MTJkYmU5YTMxN2ZmOSJ9&session_state=8e27d559-7193-4e4c-844a-2e8a6d9e566a&code=4be919c8-0e59-467e-8064-44a7b7da9a33.8e27d559-7193-4e4c-844a-2e8a6d9e566a.157d9d40-2d21-4d38-bed2-f4ddc9a2558a (173.10.0.15)
    HTTPServerRequest(protocol='https', host='test.domain', method='GET', uri='/jupyter/hub/oauth_callback?state=eyJzdGF0ZV9pZCI6ICI0ZTEyMzM3MTkxOWY0OGI3YTk2MTJkYmU5YTMxN2ZmOSJ9&session_state=8e27d559-7193-4e4c-844a-2e8a6d9e566a&code=4be919c8-0e59-467e-8064-44a7b7da9a33.8e27d559-7193-4e4c-844a-2e8a6d9e566a.157d9d40-2d21-4d38-bed2-f4ddc9a2558a', version='HTTP/1.1', remote_ip='173.10.0.15')
    Traceback (most recent call last):
      File "/usr/local/lib/python3.11/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/oauthenticator/oauth2.py", line 210, in get
        user = await self.login_user()
               ^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyterhub/handlers/base.py", line 928, in login_user
        authenticated = await self.authenticate(data)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/jupyterhub/auth.py", line 493, in get_authenticated_user
        authenticated = await maybe_future(self.authenticate(handler, data))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/oauthenticator/oauth2.py", line 1031, in authenticate
        user_info = await self.token_to_user(token_info)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/oauthenticator/oauth2.py", line 946, in token_to_user
        return await self.httpfetch(
               ^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/oauthenticator/oauth2.py", line 688, in httpfetch
        return await self.fetch(
               ^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/oauthenticator/oauth2.py", line 654, in fetch
        raise e
      File "/usr/local/lib/python3.11/site-packages/oauthenticator/oauth2.py", line 633, in fetch
        resp = await self.http_client.fetch(req, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    tornado.httpclient.HTTPClientError: HTTP 403: Forbidden
    
[E 2024-06-20 10:59:04.240 JupyterHub log:184] {
      "Cookie": "_xsrf=[secret]; oauthenticator-state=[secret]",
      "Priority": "u=1",
      "Sec-Fetch-User": "?1",
      "Sec-Fetch-Site": "none",
      "Sec-Fetch-Mode": "navigate",
      "Sec-Fetch-Dest": "document",
      "Upgrade-Insecure-Requests": "1",
      "Sec-Gpc": "1",
      "Dnt": "1",
      "Accept-Encoding": "gzip, deflate, br, zstd",
      "Accept-Language": "en-US,en;q=0.5",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:127.0) Gecko/20100101 Firefox/127.0",
      "X-Scheme": "https",
      "X-Forwarded-Scheme": "https",
      "X-Forwarded-Proto": "https,http",
      "X-Forwarded-Port": "443,80",
      "X-Forwarded-Host": "test.domain",
      "X-Forwarded-For": "173.10.0.15,::ffff:172.16.0.71",
      "X-Real-Ip": "173.10.0.15",
      "X-Request-Id": "1a913d3b7d1353899faa3f349c535e11",
      "Host": "test.domain",
      "Connection": "keep-alive"

it fails at the redirect oauth callback. What/where could be the issue?

keycloak version: 22.0.5-debian-11-r4 and the chart 17.3.6

I was abled to fix the issue.
I added openid to the client scope in keycloak and then used this config.

  config:
    JupyterHub:
      authenticator_class: generic-oauth
    Authenticator:
      enable_auth_state: true
      allow_all: true
    GenericOAuthenticator:
      client_id: "my-id"
      client_secret: "32382340843"
      oauth_callback_url: https://site.domain/jupyter/hub/oauth_callback
      authorize_url: https://site.domain/auth/realms/ui/protocol/openid-connect/auth
      token_url: https://site.domain/auth/realms/ui/protocol/openid-connect/token
      userdata_url: https://site.domain/auth/realms/ui/protocol/openid-connect/userinfo
      login_service: keycloak
      username_claim: email
      tls_verify: false
      userdata_params:
        state: state
      scope:
        - openid