JupyterHub Notebook 500 : Internal Server Error, Redirect loop detected

Hello,

Upon attempting to access a user notebook, a redirection loop occurs between paths:

  • /jupyterhub/user/admin//jupyterhub/hub/user/admin/
  • /jupyterhub/hub/user/admin//jupyterhub/user/admin/?redirects=1
  • The loop leads to a 500 error with Redirect loop detected on /jupyterhub/hub/user/admin/?redirects=1.

Setup:
I have deployed JupyterHub on Kubernetes using the latest official Helm Chart (
JupyterHub 5.2.1). I did only the minimal configuration to make it work on a private secured Kubernetes.
There’s only 1 user “admin” with no password, no database, no https. The proxy is configured with a secretToken (openssl rand -hex 32), I disabled features requiring webhooks and pvc.
For my notebook, i’m using the “minimal-notebook:notebook-7.2.2”.
There’s no issue in the hub nor the notebook pod when spawning.
I can, if needed provide all the overwrite values.

Access to JupyterHub is through an SSO proxy configured with:

  • backend_url: http://hub.data:8081/jupyterhub
  • path_prefix: /jupyterhub/
    I don’t think the SSO is related to the redirection issues as I have the same issue going through the webbrowser with SSO or directly with curl with a token on http://hub.data:8081/jupyterhub from a pod.

The Hub is working well, I can spawn my notebook, but when I want to connect on my server, I get the Redirect loop detected with theses logs:
[I JupyterHub users:899] Server admin is ready
[I JupyterHub log:192] 200 GET /jupyterhub/hub/api/users/admin/server/progress?_xsrf=[secret] (admin@ip)
[D JupyterHub scopes:1010] Checking access to /jupyterhub/hub/spawn-pending/admin via scope servers!server=admin/
[I JupyterHub log:192] 302 GET /jupyterhub/hub/spawn-pending/admin?_xsrf=[secret] → /jupyterhub/user/admin/ (admin@IP)
[I JupyterHub log:192] 302 GET /jupyterhub/user/admin/ → /jupyterhub/hub/user/admin/ (@IP)
[I JupyterHub log:192] 302 GET /jupyterhub/hub/user/admin/ → /jupyterhub/user/admin/?redirects=1 (admin@IP)
[I JupyterHub log:192] 302 GET /jupyterhub/user/admin/?redirects=1 → /jupyterhub/hub/user/admin/?redirects=1 (@IP)
[W JupyterHub base:1844] Redirect loop detected on /jupyterhub/hub/user/admin/?redirects=1
[D JupyterHub log:192] 200 GET /jupyterhub/hub/health (@IP)
[D JupyterHub log:192] 200 GET /jupyterhub/hub/health (@IP)
[D JupyterHub proxy:925] Proxy: Fetching GET http://proxy-api:8001/api/routes
[D JupyterHub proxy:996] Omitting non-jupyterhub route ‘/’
[D JupyterHub proxy:392] Checking routes
[I JupyterHub log:192] 302 GET /jupyterhub/hub/user/admin/?redirects=1 → /jupyterhub/user/admin/?redirects=2 (admin@IP)
[I JupyterHub log:192] 302 GET /jupyterhub/user/admin/?redirects=2 → /jupyterhub/hub/user/admin/?redirects=2 (@IP)
[W JupyterHub base:1844] Redirect loop detected on /jupyterhub/hub/user/admin/?redirects=2
[D JupyterHub reflector:390] pods watcher timeout
[D JupyterHub reflector:304] Connecting pods watcher
[D JupyterHub reflector:390] events watcher timeout
[D JupyterHub reflector:304] Connecting events watcher
[I JupyterHub log:192] 302 GET /jupyterhub/hub/user/admin/?redirects=2 → /jupyterhub/user/admin/?redirects=3 (admin@IP) [I JupyterHub log:192] 302 GET /jupyterhub/user/admin/?redirects=3 → /jupyterhub/hub/user/admin/?redirects=3 (@IP)
[W JupyterHub base:1844] Redirect loop detected on /jupyterhub/hub/user/admin/?redirects=3
[D JupyterHub log:192] 200 GET /jupyterhub/hub/health (@IP)
[D JupyterHub log:192] 200 GET /jupyterhub/hub/health (@IP)
[D JupyterHub log:192] 200 GET /jupyterhub/hub/health (@IP)
[I JupyterHub log:192] 302 GET /jupyterhub/hub/user/admin/?redirects=3 → /jupyterhub/user/admin/?redirects=4 (admin@IP)
[I JupyterHub log:192] 302 GET /jupyterhub/user/admin/?redirects=4 → /jupyterhub/hub/user/admin/?redirects=4 (@IP)
[W JupyterHub web:1873] 500 GET /jupyterhub/hub/user/admin/?redirects=4 (IP): Redirect loop detected.
[D JupyterHub base:1519] Using default error template for 500

It seems that the issues came from an internal routing. But I did not change the routing. For now the hub configuration is:
jupyterhub:
hub:
baseUrl: “/jupyterhub”
config:
JupyterHub:
debug: true

I tried playing with many parameter in “JupyterHub” config but did not work and I’m out of solutions.
The Notebook seems to be working but cannot be accessed through the WebUI or Curl because of this redirection. Because of that it does not seems to be related to the external SSO proxy.

Could it be an issue with the new version? How could I fix this?
I can provide more information if needed

Thank you

jupyterhub on port 8081 is for the internal JupyterHub API only. Everything else must go via configurable-http-proxy on port 8080(most likelyhttp://proxy.data:8080` based on your above config snippets- check the k8s service).

2 Likes

Hello @manics

Thank you for your input. I’ve changed the configuration to “proxy-public” on port 80 like the default service. (http://proxy-public.data:80). After a restart it solved the issue.