Getting Error in websocket Error: Unexpected server response: 403 when connecting

JupyterHub
1

Hi!

I am running in z2jh and I am trying to connect from vs code studio via jupyter/jupyterhub extension. It used to work, but now I (not very cleverly) updated bunch of things and now I am getting e.g.:

 notebook [I 2024-12-13 14:11:23.665 SingleUserLabApp log:192] 200 GET /user/kotra/api/kernelspecs?1734099083137 (kotra@10.224.0.6) 1.31ms                                                                    │
│ notebook [I 2024-12-13 14:11:24.430 SingleUserLabApp log:192] 200 GET /user/kotra/api/kernelspecs?1734099084142 (kotra@10.224.0.5) 1.31ms                                                                    │
│ notebook [W 2024-12-13 14:11:24.764 SingleUserLabApp decorator:136] Couldn't authenticate WebSocket connection                                                                                                 │
│ notebook [I 2024-12-13 14:11:24.764 SingleUserLabApp _xsrf_utils:125] Setting new xsrf cookie for b':7IvsQghEhb-9uA-kQGojRWrrYjT7Lrb5uIuVp0UNHQg=' {'path': '/user/kotra/', 'max_age': 3600}                  │
│ notebook [W 2024-12-13 14:11:24.765 SingleUserLabApp log:192] 403 GET /user/kotra/api/kernels/ea4ef2db-f15f-476c-be3a-cbe04d354914/channels?session_id=b5c4cdd6-8466-4429-9ff8-1d8da9b6fe7e&token=[secret] (@ │
│ notebook [W 2024-12-13 14:11:25.130 SingleUserLabApp decorator:136] Couldn't authenticate WebSocket connection                                                                                                 │
│ notebook [I 2024-12-13 14:11:25.130 SingleUserLabApp _xsrf_utils:125] Setting new xsrf cookie for b':XFHN3XAs4cMdrMgpHYy017yAmhsF74POjP8kQ6V7KIs=' {'path': '/user/kotra/', 'max_age': 3600}                  │
│ notebook [W 2024-12-13 14:11:25.131 SingleUserLabApp log:192] 403 GET /user/kotra/api/kernels/ea4ef2db-f15f-476c-be3a-cbe04d354914/channels?session_id=b5c4cdd6-8466-4429-9ff8-1d8da9b6fe7e&token=[secret] (@ │
│ stream closed EOF for jupyterhub/jupyter-kotra (block-cloud-metadata)                                                                                                                                         │
│ notebook [W 2024-12-13 14:11:25.693 SingleUserLabApp decorator:136] Couldn't authenticate WebSocket connection                                                                                                 │
│ notebook [I 2024-12-13 14:11:25.694 SingleUserLabApp _xsrf_utils:125] Setting new xsrf cookie for b':XFHN3XAs4cMdrMgpHYy017yAmhsF74POjP8kQ6V7KIs=' {'path': '/user/kotra/', 'max_age': 3600}                  │
│ notebook [W 2024-12-13 14:11:25.694 SingleUserLabApp log:192] 403 GET /user/kotra/api/kernels/ea4ef2db-f15f-476c-be3a-cbe04d354914/channels?session_id=b5c4cdd6-8466-4429-9ff8-1d8da9b6fe7e&token=[secret] (@ │
│ notebook [I 2024-12-13 14:11:26.100 SingleUserLabApp log:192] 200 GET /user/kotra/api/terminals?1734099086024 (kotra@10.224.0.7) 0.90ms                                                                      │
│ notebook [I 2024-12-13 14:11:26.384 YDocExtension] Processed 16 Y patches in one minute                                                                                                                        │
│ notebook [I 2024-12-13 14:11:26.384 YDocExtension] Connected Y users: 2                                                                                                                                        │
│ notebook [I 2024-12-13 14:11:26.830 SingleUserLabApp log:192] 200 GET /user/kotra/lab/api/workspaces?1734099086752 (kotra@10.224.0.7) 2.36ms                                                                 │
│ notebook [W 2024-12-13 14:11:28.097 SingleUserLabApp decorator:136] Couldn't authenticate WebSocket connection                                                                                                 │
│ notebook [I 2024-12-13 14:11:28.097 SingleUserLabApp _xsrf_utils:125] Setting new xsrf cookie for b':XFHN3XAs4cMdrMgpHYy017yAmhsF74POjP8kQ6V7KIs=' {'path': '/user/kotra/', 'max_age': 3600}                  │
│ notebook [W 2024-12-13 14:11:28.098 SingleUserLabApp log:192] 403 GET /user/kotra/api/kernels/ea4ef2db-f15f-476c-be3a-cbe04d354914/channels?session_id=b5c4cdd6-8466-4429-9ff8-1d8da9b6fe7e&token=[secret] (@ │
│ notebook [I 2024-12-13 14:11:28.891 SingleUserLabApp log:192] 200 GET /user/kotra/api/contents?content=1&hash=0&1734099088818 (kotra@10.224.0.7) 5.13ms                                                      │
│ notebook [I 2024-12-13 14:11:30.747 SingleUserLabApp log:192] 200 GET /user/kotra/api/kernels?1734099090675 (kotra@10.224.0.7) 1.16ms                                                                        │
│ notebook [I 2024-12-13 14:11:30.748 SingleUserLabApp log:192] 200 GET /user/kotra/api/sessions?1734099090677 (kotra@10.224.0.7) 0.88ms                                                                       │
│ notebook [W 2024-12-13 14:11:30.934 SingleUserLabApp decorator:136] Couldn't authenticate WebSocket connection                                                                                                 │
│ notebook [I 2024-12-13 14:11:30.934 SingleUserLabApp _xsrf_utils:125] Setting new xsrf cookie for b':jIvsXH4IcLJT3gdEgHZZz49XqG5HCdqdrf8tZ-gQ8L8=' {'path': '/user/kotra/', 'max_age': 3600}                  │
│ notebook [W 2024-12-13 14:11:30.935 SingleUserLabApp log:192] 403 GET /user/kotra/api/kernels/ea4ef2db-f15f-476c-be3a-cbe04d354914/channels?session_id=b5c4cdd6-8466-4429-9ff8-1d8da9b6fe7e&token=[secret] (@

and coorespondingly on the client I see:

13:54:04.444 [error] Error in websocket: (403) Forbidden, undefined, Headers = {"content-length":"2984","content-type":"text/html","date":"Fri, 13 Dec 2024 12:54:04 GMT","server":"TornadoServer/6.4.2","set-cookie":["_xsrf=MnwxOjB8MTA6MTczNDA5NDQ0NHw1Ol94c3JmfDYwOk9rcDVXV3RwVldJNFRWWm5ha1ZPYlZkalpWaE5WRUpNY1VOTGJrVXdSSGRoZWpCWFZFSkhaMnQ0VWxrOXxhZjBmMWI0NmY0ODA5NGExYWRkNDMxNDEyYjcyYjlmYmVhNWI4ZjU4Mzg4MTYzYjViN2NlNmQzNWU5ZTk0Zjll; Max-Age=3600; Path=/user/kotra/"]}
13:54:04.445 [error] Error in websocket Error: Unexpected server response: 403
    at ClientRequest.<anonymous> (/Users/~/.cursor/extensions/ms-toolsai.jupyter-2024.8.1-darwin-arm64/dist/extension.node.js:24:27806)
    at ClientRequest.emit (node:events:531:35)
    at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:698:27)
    at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)
    at TLSSocket.socketOnData (node:_http_client:540:22)
    at TLSSocket.emit (node:events:519:28)
    at addChunk (node:internal/streams/readable:559:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
    at TLSSocket.Readable.push (node:internal/streams/readable:390:5)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:191:23)

Thanks for any help!

2

Could you tell us which versions of JupyterHub/JupyterLab you were using before (when it was working) and which versions you are now (after upgrade)?

Also, how do you connect to running notebook from VSCode? I mean what URL are you using to connect?

1 Like
3

It’s version z2jh 4.0.0 JupyterHub helm charts | JupyterHub’s Helm chart repository which uses Jupyterhub 5.2.1. It was working on 3.3.8 which has Jupyterhub 4.1.6.

Also, how do you connect to running notebook from VSCode? I mean what URL are you using to connect?

I know this is a pretty bad answer, but “whatever the extension is using”. When i am setting up the jupyterhub kernels, I put in my main domain this is deployed on (behind traefik reverse proxy). I was trying to disect from the extension which endpoint it’s using, but unsucessfully: vscode-jupyter-hub/src/jupyterHubApi.ts at 5d684bbee59ab8911e99e89a08a7adb4600df3b8 · microsoft/vscode-jupyter-hub · GitHub .

I suspect from the logs on both sides that the path it’s trying to estabilish a websocket connection on /user/{username}/ path, and that’s where it’s failing.

4

I am sorry, when you asked me about what URL I used, I googled for the different types and found this: Connect to a JupyterHub from Visual Studio Code | by yuvipanda | Jupyter Blog

And with that, it works… I.e. https://my-server.com/user/username?token=… just works.

Therefore, I think it’s the extension’s fault. I am sorry, I am going to resolve/close this.

Thanks for rubberducking!

2 Likes