JupyterHub and Keycloak on localhost?

JupyterHub Zero to JupyterHub on Kubernetes
1

Hello friends, I have a perfectly working installation running on a public url. For local development I try to use localhost but the authentication fails because of DNS problems. I tried a rewrite rule in coredns without luck.

I use the official configuration

Do you have advice for local development in Kubernetes or a local example for JupyterHub and Keycloak? Is there another IAM like Keycloak that works with JupyterHub locally?

2

What are your IAM requirements for development? If it’s just to login to JupyterHub you could use the dummy authenticator which is built in

3

Yeah thank you, I like to keep dev/prod as similar as possible.

When I clear the cookies I can login to JupyterHub with Keycloak!
When I logout of JupyterHub I get logged out of Keycloak.
I can login again with Keycloak but JupyterHub reports 500 : Internal Server Error and tornado.httpclient.HTTPClientError: HTTP 401: Unauthorized.

Keycloak logs type=USER_INFO_REQUEST_ERROR, error=invalid_token, auth_method=validate_access_token.

Now I have to clear the cookies again for JupyterHub to work.
The logout_redirect_url works correctly, but I don’t know why JupyterHub can’t logout, do you?

4

Which cookies are you clearing, JupyterHub’s or Keycloak’s?

5

I clear all cookies in the browser so I can visit JupyterHub again. It forwards to Keycloak and after the Keycloak login a JupyterLab is launched.

I found a log saying that my JupyterLab expects JuypterHub 3.0.0 but I use JupyterHub 3.1.1. Could this be the cause of this problem?

6

Can you turn on debug logging in JupyterHub, and show us the full logs for logout and login, along with the Keycloak logs?

It might be useful to see your full config too.

7

I tried some more things and noticed that the logout problem has nothing to do with browser cookies.
The workaround is not about clearing all cookies (Keycloak and JupyterHub) but all browsing data. My browser probably stores some additional information.

Kubernetes on localhost is difficult because all internal communication urls need to be rewritten in coredns, otherwise they will point to a containers own loopback device.

I come to the conclusion that my issue is not related to JupyterHub. Thank you for the support.

8

I need help. My JupyterHub config

GenericOAuthenticator:
  client_id: []
  client_secret: []
  oauth_callback_url: http://jupyter.localhost/hub/oauth_callback
  authorize_url: http://keycloak.localhost/realms/local/protocol/openid-connect/auth
  token_url: http://keycloak.localhost/realms/local/protocol/openid-connect/token
  userdata_url: http://keycloak.localhost/realms/local/protocol/openid-connect/userinfo
  logout_redirect_url: https://keycloak.localhost/realms/local/protocol/openid-connect/logout?post_logout_redirect_uri=https://localhost&client_id=[]
  login_service: 'keycloak'
  username_key: 'preferred_username'
  userdata_params:
    state: state
  scope: ['openid']

This is Keycloak

[org.keycloak.events] (executor-thread-171) type=USER_INFO_REQUEST_ERROR, realmId=[], clientId=null, userId=null, ipAddress=10.42.1.80, error=invalid_token, auth_method=validate_access_token

That is JupyterHub

[I JupyterHub oauth2:102] OAuth redirect: 'http://jupyter.localhost/hub/oauth_callback'
[D JupyterHub base:559] Setting cookie oauthenticator-state: {'httponly': True, 'expires_days': 1}
[I JupyterHub log:186] 302 GET /hub/oauth_login?next=%2Fhub%2F -> http://keycloak.localhost/realms/local/protocol/openid-connect/auth?response_type=code&redirect_uri=http%3A%2F2Fjupyter.localhost%2Fhub%2Foauth_callback&client_id=[]&state=[secret]&scope=openid (@10.42.0.1) 1.46ms
[D JupyterHub log:186] 200 GET /hub/health (@10.42.1.1) 1.51ms
[D JupyterHub log:186] 200 GET /hub/health (@10.42.1.1) 1.53ms
[D JupyterHub reflector:362] pods watcher timeout
[D JupyterHub reflector:281] Connecting pods watcher
[D JupyterHub reflector:362] events watcher timeout
[D JupyterHub reflector:281] Connecting events watcher
[E JupyterHub oauth2:386] Error fetching user data 401 GET http://keycloak.localhost/realms/local/protocol/openid-connect/userinfo:
[E JupyterHub web:1798] Uncaught exception GET /hub/oauth_callback?state=[]&session_state=[]&code=[] (10.42.0.1)
    HTTPServerRequest(protocol='http', host='jupyter.localhost', method='GET', uri='/hub/oauth_callbackstate=[]&session_state=[]&code=[]', version='HTTP/1.1', remot
e_ip='10.42.0.1')
    Traceback (most recent call last):
      File "/opt/bitnami/miniconda/lib/python3.8/site-packages/tornado/web.py", line 1713, in _execute
        result = await result
      File "/opt/bitnami/miniconda/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 222, in get
        user = await self.login_user()
      File "/opt/bitnami/miniconda/lib/python3.8/site-packages/jupyterhub/handlers/base.py", line 801, in login_user
        authenticated = await self.authenticate(data)
      File "/opt/bitnami/miniconda/lib/python3.8/site-packages/jupyterhub/auth.py", line 491, in get_authenticated_user
        authenticated = await maybe_future(self.authenticate(handler, data))
      File "/opt/bitnami/miniconda/lib/python3.8/site-packages/oauthenticator/generic.py", line 165, in authenticate
        user_data_resp_json = await self._get_user_data(token_resp_json)
      File "/opt/bitnami/miniconda/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 387, in fetch
        raise e
      File "/opt/bitnami/miniconda/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 366, in fetch
        resp = await self.http_client.fetch(req, **kwargs)
    tornado.httpclient.HTTPClientError: HTTP 401: Unauthorized

[D 33.112 JupyterHub base:1342] No template for 500

This only occurs after the second Login and is not related to cookies.

Can you tell which parts of the login flow involve cluster interal communication and which external? I probably forgot a forwarding rule?

If you have a working localhost solution, please share.

9

I’ve got an example in

running everything on a single node K8s cluster using the external IP of the host.