We have Zero to JupyterHub on Kubernetes running on an on-prem Kubernetes cluster; ingresses are managed by Kong. Secure WebSocket connections to kernels are failing with no discernible error on behalf of JupyterHub. Based on my research, I suspect that our Kong ingress is not forwarding or upgrading WSS connections, or that I need to modify the header values sent from JupyterHub’s Proxy to include ‘allow-origin: *’. I’ve also read that it may help to downgrade Tornado to 5.x. I currently do not have HTTPS enabled at the proxy level, and was hoping to keep that as-is due to our complicated on-prem cert behavior. I’ve also read that it may help to set the Hub’s bind_url to just local loopback so it’s not listening on other addresses.
Can anyone say whether these are applicable approaches, or should I really be looking at our Kong implementation (which I do not control)?
The single user pod (minimal-notebook, but I have tried others) makes the WebSocket connection to the kernel api. The kernelmanager gets buffer for the connection, but then immediately clears the buffer.
Update: I’m not sure if the WSS issue is from the browser or between the hub and the singleuser pod. The WSS connection from Firefox targets /channels?session_id, and I see the singleuser pod open this connection, however the error that eventually comes back says that “Firefox cannot establish a WSS connection to /api/kernels”. Very confused as to what’s really going on here.