400 : Bad Request OAuth state missing from cookies

JupyterHub
1

I have setup two services of jupyterhub on single server which are running on different port
root@e2e-94-254:~# netstat -lntup | grep node
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 683069/node
tcp 0 0 0.0.0.0:20000 0.0.0.0:* LISTEN 682679/node
tcp 0 0 127.0.0.1:20002 0.0.0.0:* LISTEN 682679/node
tcp 0 0 127.0.0.1:8001 0.0.0.0:* LISTEN 683069/node

I have setup nginx as proxy, when any one service among two running it working fine users can creates notebook and work very easily, but when we start both service at a time the facing an issue
400 : Bad Request

OAuth state missing from cookies

logs of one serive running on 20000

[D 2023-12-08 18:26:33.191 JupyterHub proxy:880] Proxy: Fetching GET http://127.0.0.1:20002/api/routes
18:26:33.195 [ConfigProxy] info: 200 GET /api/routes
[D 2023-12-08 18:26:33.211 JupyterHub proxy:392] Checking routes
[D 2023-12-08 18:27:30.357 JupyterHub log:186] 304 GET /hub/login?next=%2Fhub%2F (@110.224.114.130) 7.40ms
[W 2023-12-08 18:27:33.757 JupyterHub web:1869] 400 GET /hub/oauth_callback?code=a4cc893e629ce2a8749c60ad9a752d6a9b2b3e9a1db4407967e95b586882770e&state=eyJzdGF0ZV9pZCI6ICJkZTkyYjM0YjFhZDc0NmVjYThhMjg1YzZmMWQ1NTEzNCJ9 (110.224.114.130): OAuth state missing from cookies
[D 2023-12-08 18:27:33.757 JupyterHub base:1351] No template for 400
[W 2023-12-08 18:27:33.759 JupyterHub log:186] 400 GET /hub/oauth_callback?code=[secret]&state=[secret] (@110.224.114.130) 3.55ms

Logs of second service running on 8000

[D 2023-12-08 18:25:05.825 JupyterHub proxy:392] Checking routes
[I 2023-12-08 18:27:30.246 JupyterHub log:186] 302 GET /hub/ → /hub/login?next=%2Fhub%2F (@110.224.114.130) 3.34ms
[I 2023-12-08 18:27:32.558 JupyterHub oauth2:97] OAuth redirect: https://mlflowtest.appice.io/hub/oauth_callback
[D 2023-12-08 18:27:32.559 JupyterHub base:568] Setting cookie oauthenticator-state: {‘httponly’: True, ‘expires_days’: 1}
[I 2023-12-08 18:27:32.561 JupyterHub log:186] 302 GET /hub/oauth_login?next=%2Fhub%2F → https://gitlab.com/oauth/authorize?response_type=code&redirect_uri=https%3A%2F%2Fmlflowtest.appice.io%2Fhub%2Foauth_callback&client_id=7df3d669646c2bc621fdea1db9d815b7febf1bddbb4e2d20cae48eff23f40683&state=[secret] (@110.224.114.130) 4.02ms

Please give solution if any one knows

2

Are you attempting to access both instances of JupyterHub from the same browser session? If so, that explains the reason. You are mixing the session cookies from both instances of JupyterHub.

Try to access each of them in a separate incognito mode session.

3

Hi @mahendrapaipuri , Yes, I have setup the nginx to divert request on these two instances as per condition, intention behind this that fault tolerance will be there so clients will not face any issues while one instance goes down. below is my nginx configuration

location / {


    proxy_pass http://jupyter_backend;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

upstream jupyter_backend
{
server 216.48.177.254:8000;
server 216.48.177.254:20000;

}

4

I dont think this config would work. JupyterHub does not support clustered mode operation which is what you are attempting here. Also JupyterHub uses a lot of in memory caching for DB operations so sharing a single DB instance between two JupyterHub instances wont ensure consistency.

You can try active-passive approach with something like keepalived to replace a failed JupyterHub instance by a standby instance.

1 Like
5

Thanks for the suggestion @mahendrapaipuri , I will check this.